If a student were to have no knowledge of the subject matter and is to walk into an exam having to answer every question, the likelihood of them scoring highly would be very low. The probability of the student passing the exam would be heavily dependent on the question style of the exam. This report investigates different question styles and scoring methods that form an exam; looking specifically at multiple choice, multiple response, matching questions and how dichotomous and polytomous scoring methods affect the expected score of the student. Of the cases investigated in this report, the one which would give the lowest probability of passing an exam purely by guessing would be a multiple choice paper where each question has 1 correct response from 4 answer options. The results conclude that, from the cases studied, a multiple choice exam paper with 2 options would give the highest probability of the student passing the exam. An exam that adopts a polytomous scoring method would give a higher probability of passing an exam over a dichotomous method.
2. Introduction
When designing an exam, the examiner aims to make the exam so that a student is unable to pass purely by guessing each response. It is assumed that the participant is thinking only using the information given in the question, answering every question and does not select every response available in their answer. If the student selects more than the correct number of responses, the answer has been counted as incorrect and scores 0 marks.
The background of this report explains how different question types can be categorised into mathematically similar groups. The groups being investigated in this report include multiple choice, multiple response and matching questions. Examining multiple choice questions, it is easy to understand that students have a higher probability of being able to guess the correct answer based on 1 of 4 answers being correct, with probability of scoring a correct answer of 0.25, as opposed to an essay type/numerical calculation question. The probability of guessing an essay type question is set to zero. An exam paper of multiple choice questions has a binomial distribution. This report explores how the number of answer options alters the probability of passing an exam, where the pass mark is 40%.
Multiple response questions are similar to multiple choice, only they have more than one correct response; however, they differ mathematically when there are a known or unknown number of correct responses. These are explored within this report. Dichotomous and polytomous scoring methods have also been studied to see the effect on the random guess scores, and the probability of the student passing the exam in the cases investigated. The effect of using each choice once or multiple times for matching questions has also been considered within the report, for both dichotomous and polytomous scoring.
The discussion includes analysis of the results, examining the simulations run for each type of question. This includes a discussion, of which type of scoring method leads to the lowest probability of passing by guessing, which is required for an exam testing a students knowledge.
3. Background
There are numerous question types (elearn.southampton.ac.uk, (2015)), which can be grouped into mathematically similar groups. The first group consists of multiple choice, true/false, either/or and hot spot – all requiring the student to select an answer from a given list. These are grouped as the simplest examples.
Matching and multiple response questions are then grouped together, where they can come in many variations. The simplest case is where all items are matched only once, gaining a score of 1 for all correct responses. Since there are many combinations of responses, it is then more difficult than multiple choice questions to pass purely by guessing.
Essay based, short answer, numerical calculation and fill in the blanks are too complicated to calculate their random guess score as there is not enough detail about each one to be computed mathematically. In these cases, the participant would require some prior knowledge, as they are not given enough additional information to answer. The probability of passing these exams with no prior knowledge would be 0 (or very close to it). These types of exam papers would test knowledge the most, as there is the lowest probability of passing by guessing.
3.1 Monte Carlo Methods
In statistics, the Monte Carlo method (Boyle, Phelim P. (1977)), was known as ‘model sampling’ and was used to verify properties of estimates by mimicking the settings for which they were designed. Monte Carlo sampling became prominent in the 1940s/early 1950s. It was used to solve problems in physics related to atomic weapons. Many problems were modelled deterministically – now it is standard to use random sampling on problems stated deterministically.
The purpose of Monte Carlo Sampling is to predict performance without building hundreds of samples or conducting lots of experiments. It relies heavily on repeated random sampling, generally approximating solutions. It is used most often in cases where analytical or numerical solutions do not exist or are to complicated to calculate.
The Monte Carlo method focuses on estimating the expectation of a function using a range of samples, where it is aimed to evaluate the following: Let g(y) be some function, where Y1,…, YN are independent and identically distributed, of a sample of size N for discrete variables,
E[g(Y)]=∑_(y=0)^n▒g(y)p(y) , (Chung, Kai L (1974))
Taking an estimate of g(Y) to be 1/N ∑_(i=1)^N▒〖g(Y_i)〗 we can show the following properties:
Unbiased
E(1/N ∑_(i=1)^N▒g(Y_i ) )=E(g(Y)),
(by linearity of expectation, (Meyer, Albert R and Rubinfeld, Ronitt (2005))),
Variance tends to 0 as N tends to ∞
Var(1/N ∑_(i=1)^N▒g(Y_i ) )=1/N×Var(g(Y))
(by consistency, (statlect.com (2017)))
Standard deviation tends to 0 as N tends to ∞
SD(1/N ∑_(i=1)^N▒g(Y_i ) )=√(Var(1/N ∑_(i=1)^N▒g(Y_i ) ) )=1/√N SD(g(Y))
The standard deviation shown in formula (3) presents the error and therefore the error decreases proportionally as 1/(√N), where N is the total number of exam papers investigated, e.g. the number of times the simulation has been run.
There are 4 main steps to Monte Carlo sampling, as follows:
Determine the statistical properties of the possible inputs
Generate a set of random data, that follow the statistical properties above
Deterministic calculation
Statistically analyse the results
Example 3.1
For a multiple choice question, the steps are as follows:
Identify a binomial distribution (n, p), where n is the number of questions and p is the probability of getting an answer correct.
Generate some random data in R
Calculate the average expected score and variance
Compare results to exact calculations, determine the probability of passing the exam by guessing.
The random guess score, R_GS, is a mathematical statistic, (Docs.moodle.org (2017)), illustrating the mean score students would expected to achieve purely by guessing. This is only applicable for questions that are mathematically similar to multiple choice questions, such as belonging to the same statistical distribution, hence cannot be computed for essay based questions.
The random guess score throughout assumes that this sco
re is obtained with no prior knowledge on the subject matter and the question is answered random
ly, with no logic behind each response. The participant is thinking using only the information in the question and the examiner wants to ensure a participant cannot pass the exam with no knowledge. Throughout, it is assumed that the student answers every question on the paper and does not select every answer in their response, as this could be counted as a way of cheating. If the student selects more than the correct number of responses, the answer has been counted as incorrect and scores 0 marks.
4. Multiple choice
Students are told there is 1 correct answer per m responses. Taking X to be the score of the question,
X~Bernoulli(p=1/m)
When looking at an entire exam paper, of n questions, this then becomes a Binomial distribution as follows:
Y~Binomial(n,1/m),where Y=∑_(i=1)^n▒X_i
Hence, the expected score and variance are (Snell, James L and Grinstead, Charles M (2006)):
R_GS=E(Y)=n/m
Var(Y)=np(1-p)
With a pass mark of (k*100)%, the calculation of P(Y≥kn) gives the probilabity of passing the exam (stattrek.com (2017)), where
P(Y=kn)=(■(n@kn)) p^kn (1-p)^(n-kn)
Calculating exact, an exam paper with 50 questions, with 4 responses, one of which is correct with a pass mark of 40%, the probability of success, p, would be 0.25. This then calculates the expected score (equation 3), E(Y) =12.5. The probability of passing the 40% pass mark is then calculated as follows:
P(Y≥20)=1-P(Y<20)=1-∑_(i=1)^19▒(■(50@20)) (0.25)^20 (0.75)^30
=1-0.9860823…=0.013918 (5sf)
The binomial probability mass function graph (figure 1), illustrates this mathematical data. The chances of the participant scoring above 40% (a score of 20) are very unlikely. The expected score is shown to be approximately 12.5, aligning with the above calculation.
Likewise, these calculations can be done for 20 questions and 2 options also. The summary of all hand calculated results are shown below.
50 questions 20 questions
4 responses 2 responses 4 responses 2 responses
Pass mark 20 8
RGS
12.500 25.000 5.0000 10.000
Var(Y) 9.3750 12.500 3.7500 5.0000
Table 1: Summary of statistical properties of each case
4.1 R Simulation
R (R Core Team (2015)), can be used to simulate some random data to calculate the probability of gaining 40% or above in the exams, without calculating every scenario by hand. Running a simulation 1000 times and taking an average increases the accuracy of the results given. The simulation run studied the impact of varying question numbers, either 50 or 20, and the number of responses, 4 or 2, with just one response correct.
Probability of scoring 40% or above (5sf)
4 responses 2 responses
50 questions 0.013918 0.89868
20 questions 0.040925 0.74828
Table 2: Probability of scoring 40% or above
First, studying a range of exams, by R simulation, on the impact of a varying number of questions, from 50 to 20 questions, where there are 4 answer options with 1 correct answer, summarizing the results to 5sf below:
Number of exam questions
50 20
RGS
12.511 4.9973
Var(Y) 9.3258 3.7669
Table 3: How the number of exam questions affects R_GS and Var(Y) for 4 answer options
Also, studying the number of responses, considering 4 or 2 responses, where 1 is correct, in an exam paper of 50 questions, summarized results to 5sf are shown below:
Number of responses
4 2
RGS
12.511 25.003
Var(Y) 9.3258 12.482
Table 4: How the number of responses affects R_GS and Var(Y) for 50 questions
It is evident that these random guess scores are approximate to the previous calculations from table 1, showing the simulation has given accurate results.
5. Multiple response
Figure 2: Example of a multiple response question with 4 alternatives with 3 correct answers
Multiple response questions are similar to multiple choice questions; however, there can be more than one correct answer. The scoring method can alter the mark obtained for the correct answer; this can be done in two ways.
Dichotomous – where the question is either correct or incorrect
Polytomous – where multiple marks can be obtained by the selected responses.
If the question has a known number of correct responses, it is much easier to calculate the random guess scores, as the student will give this number of responses. Suppose a student is not told the number of correct responses, we are unsure of the number of responses given by the student. This case is therefore more difficult to calculate, requiring a simulation. The investigations below take 1234 to be the correct answer for 4 correct responses, 12345 for 5 correct responses etc. and if the student selects more than the correct number of responses, the answer has been counted as incorrect and scores 0 marks.
5.1 Dichotomous scoring
This scoring method (Jordan, Helen and Draaijer, Silvester (2011)), is where the response is always marked as either correct or incorrect, regardless of its complexity or understanding in the response.
Example 5.1
Taking a 4 alternative multiple-response question, where 3 are correct. The student would select 3 of the responses. Using the dichotomous scoring method, the student would score 1 mark if all 3 are correct, and 0 otherwise. There are many combinations possible from the 4 alternatives, calculated as follows:
(■(4@3))=4!/3!(4-3)!=4
One of these combinations will lead to a correct answer, scoring 1 mark; the rest will be incorrect, scoring 0. Represent P(C_0) and P(C_1) to be the probability of scoring 0 points and 1 point respectively. The random guess score, R_GS, can then be calculated as follows:
R_GS=P(C_0 )×0+ P(C_1 )×1=3/4×0+1/4×1=0.25 points
5.2 Polytomous scoring
The participant is told to select n responses from m answer alternatives, all equally likely to be selected, with probability of n/m. The probability of selecting the first correct response,
P(C_1 )=n/m, the second P(C_2 )=n/m also, etc. The expected value of the sum equals the sum of the expected values of the random variables, being uninfluenced by dependence. Hence, we can write
R_GS=∑▒(n/m) ×(score given for selecting response C_i)
Hence, if one mark is given for each correct response, then
R_GS=n^2/m
The score given can also be negative, meaning a penalty is given for incorrect responses, including giving too many responses.
Example 5.2
Following from example 5.1, the score would change when using a polytomous scoring method. If the student receives 1 mark per correct response, and the student is giving
3 responses, where 3 of the 4 alternatives are correct. As previously calculated, there are 4 combinations possible. The random guess score can be calculated as:
R_GS=P(C_0 )×0+ P(C_1 )×1+P(C_2 )×2+P(C_3 )×3= 0
×0+0×1+3/4×2+1/4×3
=2.25 points
Which follows R_GS=n^2/m=3^2/4=2.25
5.3 What if there are an unknown number of correct responses?
If the participant was not told how many responses to give, there is a possibility they may give too many or not enough. This would lead to not attaining full marks or losing marks if a penalty was given for too many answers. The score obtained would differ for both dichotomous scoring and polytomous.
Example 5.3.1 – dichotomous
Taking a 4, m, alternative multiple-response question, where n are correct. The student may not select enough response or may select too many, this leads to an increase in the number of possible combinations of responses. Using the dichotomous scoring method, the student would score 1 mark if all n are correct, and 0 otherwise. The total number of possible combinations is calculated as follows:
∑_(n=1)^m▒m!/n!(m-n)!=∑_(n=1)^4▒4!/n!(4-n)!=15
Hence, the random guess score, with 3 answers being correct, is then
R_GS=P(C_0 )×0+ P(C_1 )×1=14/15×0+1/15×1=0.066666 points
Example 5.3.2 – polytomous
Following from example 5.3.1, the student receives 1 mark per correct response. There are 15 possible combinations of responses and therefore the random guess score, with 3 answers being correct, can be calculated by:
R_GS=P(C_0 )×0+ P(C_1 )×1+P(C_2 )×2+P(C_3 )×3
=2/15×0+6/15×1+6/15×2+1/15×3=1.4 points
5.4 Exact calculation
Number of alternatives Number of correct responses Dichotomous scoring method
Known number correct Unknown number correct
m n Random guess score Random guess score
3 1 0.33
0.33
1.00 0.14
0.14
0.14
3 2
3 3
4 1 0.25
0.17
0.25
1.00 0.07
0.07
0.07
0.07
4
4
4 2
3
4
5
5
5
5
5 1
2
3
4
5 0.2
0.1
0.1
0.2
1.00 0.03
0.03
0.03
0.03
0.03
6
6
6
6
6
6 1
2
3
4
5
6 0.17
0.07
0.05
0.07
0.17
1.00 0.02
0.02
0.02
0.02
0.02
0.02
Table 5: Random guess scores for multiple response questions by the dichotomous scoring method
The percentage achieved in the above scenario can be calculated by
〖 R〗_GS×100=% achieved in an exam
This illustrates the mark they are expected to achieve purely through guessing. The only situations above, in table 5, in which the student would be expected to pass the exam are when all the responses are correct, when you know the number required, as highlighted.
Number of alternatives Number of correct responses Polytomous scoring method
Known number correct Unknown number correct
m n Random guess score % Achieved in 50 question exam Random guess score % Achieved in 50 question exam
3
3
3 1
2
3 0.33
1.33
3.00 33.0
66.5
100 0.14
0.86
1.71 14.0
43.0
57.0
4
4
4
4 1
2
3
4 0.25
1.00
2.25
4.00 25.0
50.0
75.0
100 0.07
0.53
1.4
2.13 7.00
26.7
46.7
53.3
5
5
5
5
5 1
2
3
4
5 0.20
0.80
1.80
3.20
5.00 20.0
40.0
60.0
80.0
100 0.03
0.35
1.06
1.93
2.58 3.00
17.7
35.5
48.3
51.6
6
6
6
6
6
6 1
2
3
4
5
6 0.17
0.67
1.50
2.67
4.17
6.00 17.0
33.5
50.0
66.8
83.4
100 0.02
0.19
0.68
1.65
2.46
3.11 2.00
9.50
22.8
41.25
49.2
51.9
Table 6: Random guess scores for multiple response questions by the polytomous scoring method
The percentage achieved in the above scenario can be calculated by
〖 R〗_GS/n×100=% achieved in an exam
This illustrates the mark they are expected to achieve through guessing. The student would be expected to pass the exam, where the pass mark is 40%, in the highlighted situations above, in table 6.
6. Matching
Below is an example of a matching type question, these can also include drag and drop questions. There are 4 options with 3 markers; one of the options is a foil.
Figure 3: Example of matching question, 4 options, 3 markers
6.1 Dichotomous scoring
Assuming the participant is required to position 4 options into 3 spaces, the student receives 1 mark if all four responses are correct and 0 otherwise. Each option can only be used once. The probability of getting the first match correct is 1/4, the 2nd 1/3 etc. The random guess score is then calculated to be R_GS=1/4×1/3×1/2×1/1=0.041667
6.2 Polytomous scoring
Assume the student scores 1 mark for each correct position. The question has 4 options, 1,2,3,4, which have to be allocated to 3 markers. The number of permutations needs to be calculated in order to determine how many variations give all, 2, 1 and 0 options in the correct space – these are shown in the table below:
All in correct space 2 in correct space 1 in correct space 0 in correct space
Permutations 123 124
143
423 134
142
132
421
321 324
413
213
243 432
431
412
312
314
341 342
214
234
241
231
Total: 1 3 9 11 24
Table
7: Table showing possible permutations for matching question with 4 options, 3 markers
The random guess score can now be calculated by
R_GS=3×1/24+2×3/24+1×9/24+0×11/24=0.75 points
6.3 What if each choice can be used an unlimited number of times?
If each option can be used multiple times then the possible number of combinations and permutations increases and so the probability of selecting the correct response
s decreases.
Example 6.3.1 – dichotomous
The participant would acquire 1 mark for getting the response correct and 0 otherwise. Taking the example of selecting 4 options into 3 spaces, the probability of getting each match correct would then be equal, as ¼. The random guess score is then calculated to be:
R_GS=(1/4)^4=0.0039063
Example 6.3.2 – polytomous
Extending on from example 6.3.1, where 4 options, 1, 2, 3 and 4 have to be allocated to 3 markers, each option can now be used multiple times. The number of permutations now increases to having a total of 64. Taking the correct order to be 123, the permutations are summarised below:
All in correct space 2 in correct space 1 in correct space 0 in correct space
Permutations 123 121
122
124
223
323 423
143
113
133 111
112
114
131
132
134
141
142
144 222
224
322
324
421
422
424
321
221 333
213
413
443
243
313
343
433
233 444
211
214
231
212
232
234
241
242 244
311
312
314
331
332
334
341
342 344
411
412
414
431
432
434
441
442
Total: 1 9 27 27 64
Table 8: Table showing possible permutations for matching question with 4 options, 3 markers
Hence, the random guess score is now calculated as
R_GS=3×1/64+2×9/64+1×27/64+0×27/64=0.75 points
6.4 Exact calculation
Number of alternatives Total number of match alternatives Dichotomous scoring method
Using each choice once only Using each choice unlimited number of times
m n Random guess score Random guess score
2
2
2 2
3
4 0.50
0.17
0.04 0.25
0.038
0.0039
3
3
3 3
4
5 0.17
0.04
0.01 0.038
0.0039
0.00032
4
4
4 4
5
6 0.04
0.01
0.00 0.0039
0.00032
0.000021
5
5
5 5
6
7 0.01
0.00
0.00 0.00032
0.000021
0.0000012
Table 9: Random guess scores for matching questions by the dichotomous scoring method
The percentage achieved in the above scenario can be calculated by
〖 R〗_GS×100=% achieved in an exam
This illustrates the mark they are expected to achieve purely through guessing. There is only one situation, highlighted above, in which the student would be expected to pass the exam – where there are 2 alternatives to match to 2 others. In this case, the student would be expected to score 50%. When using each choice an unlimited number of times, the student is highly likely to score 0% by guessing.
Number of alternatives Total number of match alternatives Polytomous scoring method
Using each choice once only Using each choice unlimited number of times
m n Random guess score % Achieved in 50 question exam Random guess score % Achieved in 50 question exam
2
2
2 2
3
4 1.00
0.67
0.50 50.0
33.5
25.0 1.00
0.67
0.50 50.0
33.5
25.0
3
3
3 3
4
5 1.00
0.75
0.60 33.3
25.0
20.0 1.00
0.75
0.60 33.3
25.0
20.0
4
4
4 4
5
6 1.00
0.80
0.67 25.0
20.0
16.8 1.00
0.80
0.67 25.0
20.0
16.8
5
5
5 5
6
7 1.00
0.83
0.71 20.0
16.7
14.2 1.00
0.83
0.71 20.0
16.7
14.2
Table 10: Random guess scores for matching questions by the dichotomous scoring method
The percentage achieved in the above scenario can be calculated by
〖 R〗_GS/m×100=% achieved in an exam
This illustrates the mark they are expected to achieve through guessing. The expected scores above show there are only 2 situations where the student is expected to pass the exam. Clearly, we can see that the random guess scores for using each choice only once, and a multiple number of times are equivalent. This can be illustrated, following the case of m=2, as follows.
In the case of using each choice only once, “n” P_2 is the total number of partitions of n into 1s and 2s. The total count of options that would result in a score of 1 is given as (n-1)^2-(n-2), and there is only 1 option giving a total possible score of 2. Hence, the options giving a score of one are the remaining of the total. The random guess score is therefore given below.
R_GS=((n-1)^2-(n-2))/(“n” P_2 )×0+(“n” P_2-((n-1)^2-(n-2))-1)/(“n” P_2 )×1+1/(“n” P_2 )×2= (n^2-2n+1-n+2)/(n^2-n)×0+(n^2-n-(n^2-2n+1-n+2)-1)/(n^2-n)×1+2/(n^2-n)=(2n-2-2+2)/(n^2-n)=2(n-1)/n(n-1) =2/n
In the case of using each choice an unlimited number of times, n^2 is the total number of possible combinations where the student can repeat an alternative. The total count of options that would result in a score of 1 is given as (n-1)^2, and there is only 1 option giving a total possible score of 2. Hence, the options giving a score of one are the remaining of the total. The random guess score is therefore given below.
R_GS=(n-1)^2/n^2 ×0+(n^2-(n-1)^2-1)/n^2 ×1+1/”n” ^2 ×2
=(n^2-n^2+2n-1-1)/n^2 ×1+2/n^2 ×2=2n/n^2 =2/n
Discussion
By investigating each question type individually, it is evident that there are various ways to present each type of question. Multiple choice questions demonstrate a decrease in the probability of passing with more questions, where only one correct answer exists. The random guess score and variance are higher for a larger number of questions, which coincide with the increased raw pass mark. The simulation illustrates that the individual is more likely to score with fewer options, as expected.
For multiple response questions there is a higher percentage achieved when the exam paper adopts the polytomous scoring method. When the paper adopts the dichotomous scoring method, the student is not expected to achieve higher than 33%. An individual acquir
es a higher random guess score, through the dichotomous method, when there are a known number of correct responses. The polytomous scoring method provides a generally higher score and achieved percentage for a known number of correct responses. 72% of the cases investigated have an expected percentage above 40%, for a known number of correct responses, as opposed to 0% for dichotomous.
For matching questions, again, there is a higher random guess score for the polytomous scoring method, in every case. The only case to pass, using the polytomous scoring method is when there are 2 alternatives and 2 matching alternatives, either using each only one or using each an unlimited number of times. When using the dichotomous method, using each choice only once, this gives a higher random guess score and therefore a higher percentage achieved in the exam paper. The only case in which a student would pass is also for 2 alternatives, with 2 matching alternatives, however, only when using each choice only once. 75% of the dichotomous cases investigated score zero when using each of the choices an unlimited number of times.
Conclusion
This report explores different styles of exams, including different types of questions and different scoring methods. The investigation has led to the conclusion that an exam paper filled with multiple choice questions with 2 options would give the highest probability of the student passing the exam by guessing. A polytomous scoring method would give a higher probability of passing an exam over a dichotomous scoring method, for both multiple response and matching questions. The case investigated, which would give the lowest probability of passing an exam purely by guessing would be a multiple choice paper where each question has 1 correct response from 4 answer options.
9. Glossary
Bernoulli distribution The discrete probability for outcomes 0 or 1 (success or failure)
Binomial distribution The discrete probability for the number of successes in n independent experiments
Dichotomous scoring Receiving marks only for the full correct answer
Polytomous scoring Receiving marks for each correct answer given
R A language environment for statistical computing
Random guess score Expected score of a student guessing a question
Simulation Imitation of a situation – students responding to variations of question types
Variance The measure of dispersion of a random variable
10.References
Boyle, Phelim P (1977). Options: A Monte Carlo approach. Journal of Financial Economics
Chung, Kai L (1974). A Course in Probability Theory. Academic press, New York
Docs.moodle.org. (2017). Quiz Report Statistics – MoodleDocs. [online] Available at: https://docs.moodle.org/dev/Quiz_report_statistics [Accessed 20 Nov. 2017].
Elearn.southampton.ac.uk, (2015). Question Types in Blackboard. [online] Available at: http://elearn.southampton.ac.uk/blackboard/questiontypes/ [Accessed 13 Nov. 2017].
Jordan, Helen and Draaijer, Silvester (2011). How to Calculate the Random Guess Score of Multiple-Response and Matching Questions
Meyer, Albert R and Rubinfeld, Ronitt (2005). Mathematics for Computer Science, Massachusetts Institute of Technology
R Core Team (2015). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, URL http://www.R-project.org/.
Snell, James L and Grinstead, Charles M (2006). Introduction to Probability
Statlect.com (2017). Fundamentals of Statistics. [online] Available at: https://www.statlect.com/fundamentals-of-statistics/variance-estimation [Accessed 23 Dec. 2017]
Stattrek.com (2017). Binomial Probability Distribution. [online] Available at: http://stattrek.com/probability-distributions/binomial.aspx [Accessed 11 Dec. 2017]
.