Master of Science
Determining Knowledge of the 10 Principles of Multimedia Instruction
Introduction to the Question
The 2012 annual report from the ASTD (American Society of Training and Development) indicated U.S. organizations spent approximately $156.2 billion dollars on employee learning and development, or about $1,182 per employee. Technology-based methods accounted for 37.3 percent of all learning methods (ASTD 2012). With so much money being spent on training, organizations would benefit from knowing and using research-based information on effective instructional techniques.
There have been a great deal of empirical research studies conducted to determine how to make multimedia learning more effective. Researcher Richard E. Mayer transferred much of this research into a theory and set of principles known as the Cognitive Theory of Multimedia Learning (Mayer 2008). This theory draws upon numerous empirical studies and research both from the Science of Learning and the Science of Instruction. The Cognitive Theory of Multimedia Learning contains 10 principles which have been shown to increase learning effectiveness within multimedia lessons (Mayer 2008). This theory and the principles created from it are widely known and cited within the research community. However, these principles do not appear to be utilized by the majority of presenters or trainers. Furthermore, this literature review has not found any studies that examined the use of multimedia learning principles.
The purpose of this study is to determine if individuals within organizations (creators and consumers of multimedia learning) are able to discern effective from ineffective multimedia learning practices based on the 10 principles of multimedia instruction outlined in the Cognitive Theory of Multimedia Learning.
Study Design
An online survey will be used to investigate whether or not individuals within organizations are able to discern effective from ineffective multimedia learning practices based on the ten principles of multimedia instruction. A Qualtrics online survey has been created that includes fourteen multiple choice questions, fourteen open ended questions and three demographic questions. There is one question for each of the ten principles of multimedia instruction and an open-ended question for each of these questions. An example of one of these questions is as follows:
Which approach would you expect to lead to better learning outcomes?
(A) A multimedia lesson in which corresponding narration and animation are presented successively.
(B) A multimedia lesson in which corresponding narration and animation are presented simultaneously (words are spoken at the same time they are illustrated in the animation).
Please explain why you gave the answer you selected.
The research indicates that the correct response is ‘B’. The responses from each of the ten questions being asked will be calculated and given a score. This score will indicate whether respondents can choose the appropriate multimedia learning methods in a given scenario. This score will be the dependent variable.
The independent variables will include an examination of whether participants have delivered or created multimedia content, what their current job is, and what (if any) formal training they have in multimedia learning development. Because this study is exploratory in nature, there are no specific predictions being made before collecting data.
An online survey should be able to gather responses from many individuals. This survey will be able to reach people in many different geographic locations and from a wide variety of organizations. This should allow for a large number of respondents, but it will only be able to gather a limited amount of in-depth answers from the short answer open ended questions. This method is preferred over a qualitative interview method because a large number of responses is needed.
Study Participants
The ideal participants will be individuals who have created or consumed multimedia learning content although no responses will be eliminated from analysis. Participants will be selectively recruited from LinkedIn, Facebook, and professional contact lists. The survey link will be sent to just over 200 individuals with the hope that at least thirty people will fill out the survey. Because this study is exploratory in nature, bias is not necessarily a concern, but several demographic questions will be asked to determine what type of population is answering the survey.
Study procedures & duration
The primary method for collecting data will be an online survey that includes multiple choice and open ended questions. Participants will be asked to consent to the survey with the very first question in the survey. Participants who chose not to consent will have the survey ended immediately. The survey should take between about 8 minutes and 20 minutes depending upon the length of the answers given by the participants in the short answer questions.
Instrument/Tools
The Cognitive Theory of Multimedia Learning has generated ten principles that multimedia learning creators should follow if they want to generate better learning outcomes. This study will contain ten questions which correspond to the ten principles. Each question will present the participant with a scenario based on a principle and give two options to choose from. One option will be an incorrect approach and the other a correct approach. Participants will be asked to choose which option they believe will lead to better learning. The ten questions will then be combined to form a composite score. This composite score will become the dependent variable.
Risks and Assumptions
The primary risk for this study is getting enough participants to take the survey. This risk will be mitigated by aggressively leveraging all appropriate contact databases. There is no known risk to the participants.
I am assuming that people who create and deliver multimedia presentations have not read peer reviewed research journals on the CTML (Cognitive Theory of Multimedia Learning). The advice within the CMTL has appeared in popular text books (Clark, 2010 and Clark, 2011) that may have been read by those that create multimedia learning. If this is the case, these individuals may already know what the research indicates, even though they may not have read the journal articles on CTML.
People taking the survey might be able to guess correctly based on common sense, rather than on knowledge from the 10 principles of multimedia instruction. I am assuming I can make a test instrument that will be able to accurately identify survey respondent’s ability to discern effective from in-effective multimedia learning practices.
Outcomes
Two primary goals will be accomplished with this study. The first goal is to understand what individuals know about appropriate multimedia learning approaches. This is important because organizations are spending a great deal of money making multimedia learning. These investments should leverage the latest and greatest information available about how to create the most effective multimedia learning content.
The second goal is to determine if there are differences in participant’s knowledge of multimedia learning approaches based on the survey participant experiences within the industry and/or formal training. These factors may indicate that some individuals have more knowledge of appropriate multimedia learning approaches.
‘
References
ASTD (2012) State of the industry report (ASTD’s Annual Review of Workplace Learning and Development Data). American Society of Training and Development. Retrieved from store.astd.org.
Clark, R. C., & Lyons, C. (2010). Graphics for learning: Proven guidelines for planning, designing, and evaluating visuals in training materials. Pfeiffer.
Clark, R. C., & Mayer, R. E. (2011). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. Pfeiffer
Mayer, R. E. (2008). Applying the science of learning: Evidence-based principles for the design of multimedia instruction. American Psychologist, 63(8), 760-769.
‘
Appendix 1 Instrument/Protocol
Q1 Thank you for participating in this research study! This study is being conducted as part of the requirements for the Masters of Science in Learning and Organizational change at Northwestern University. The survey will take about 10 minutes to complete. Your responses will be anonymous and confidential. If you have any questions, the researcher’s contact information will be included at the end of the survey. Do you agree to participate?
‘ Agree (1)
‘ Disagree (2)
If Disagree Is Selected, Then Skip To End of Survey
Q3 Instructions: Please base your answers on your own experience as a developer and/or user of multimedia presentations. We are not looking for ‘correct’ answers; rather, we are interested in the range of opinions of experienced multimedia users. This is a study of how different experienced users and creators of multimedia presentations view the best ways to share multimedia information. Multimedia learning is defined as ‘Learning from words and pictures. The words can be printed text or spoken text. The pictures can be in static form, such as illustrations, photos, diagrams, charts, or maps, or in dynamic form, such as animation or video. Examples of multimedia learning include:
– watching and listening to a narrated animation
– reading a science textbook with graphics and photos conveying information
– playing an educational video game
– attending a Power-Point presentation
Have you created multimedia learning content?
‘ Yes (1)
‘ No (2)
Answer If Instructions: Please base your answers on your own … Yes Is Selected
Q4 How frequently do you create multimedia?
‘ Daily (1)
‘ 2-3 Times a Week (2)
‘ Once a Week (3)
‘ 2-3 Times a Month (4)
‘ Once a Month (5)
‘ Less than Once a Month (6)
‘ Never (7)
Answer If Instructions: Please base your answers on your own … Yes Is Selected
Q32 What do you try to include in your lessons to make them effective?
Answer If Instructions: Please base your answers on your own … Yes Is Selected
Q36 How do you determine if you created a good multimedia learning content?
Q5 Have you delivered multimedia content?
‘ Yes (1)
‘ No (2)
Q6 Have you consumed multimedia learning?
‘ Yes (1)
‘ No (2)
Answer If Have you consumed multimedia learning? Yes Is Selected
Q33 What characteristics were in the best multimedia learning lessons you have taken?
Q8 The following questions ask your opinion on different ways of presenting multimedia information. There are no hard-and-fast rules about multimedia; different users take in information in different ways. We are interested in the different ways individuals think about multimedia, and why. Please share your honest opinion and let us know why you feel the way you do.
Q10 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson that includes both content aligned to the lesson objectives and material that is not directly related, but somewhat related, to the main content. (1)
‘ A multimedia lesson that excludes all material from the lesson not directly related to the main point. (2)
Q11 Please explain why you gave the answer you selected.
Q12 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson that highlights essential words. (1)
‘ A multimedia lesson that does not include highlighting. (2)
Q13 Please explain why you gave the answer you selected.
Q14 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson that includes animation and narration. (1)
‘ A multimedia lesson that includes animation with narration and text. (2)
Q9 Please explain why you gave the answer you selected.
Q15 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson where corresponding words and pictures are presented far from each other on the page or screen. (1)
‘ A multimedia lesson where corresponding words and pictures are presented near each other on the page or screen. (2)
Q16 Please explain why you gave the answer you selected.
Q17 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson in which corresponding narration and animation are presented successively. (1)
‘ A multimedia lesson in which corresponding narration and animation are presented simultaneously (words are spoken at the same time they are illustrated in the animation). (2)
Q18 Please explain why you gave the answer you selected.
Q19 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson where narrated animation is presented in learner-paced segments (sections that the learner can advance at whatever speed he/she likes) (1)
‘ A multimedia lesson where narrated animation is presented as a continuous presentation. (2)
Q21 Please explain why you gave the answer you selected.
Q22 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson that includes narrated animation when the students do not already know the names and characteristics of essential components (1)
‘ A multimedia lesson that includes narrated animation when the students already know the names and characteristics of essential components (2)
Q23 Please explain why you gave the answer you selected.
Q24 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson that includes graphics with spoken text (1)
‘ A multimedia lesson that includes graphics with printed text (2)
Q25 Please explain why you gave the answer you selected.
Q26 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson that includes only words. (1)
‘ A multimedia lesson that includes words and pictures. (2)
Q27 Please explain why you gave the answer you selected.
Q28 Which approach would you expect to lead to better learning outcomes?
‘ A multimedia lesson that includes narration in a formal style. (1)
‘ A multimedia lesson that includes narration in a conversational style. (2)
Q29 Please explain why you gave the answer you selected.
Q3 What year were you born?
‘ 2000 (1)
‘ 1999 (2)
‘ 1998 (3)
‘ 1997 (4)
‘ 1996 (5)
‘ 1995 (6)
‘ 1994 (7)
‘ 1993 (8)
‘ 1992 (9)
‘ 1991 (10)
‘ 1990 (11)
‘ 1989 (12)
‘ 1988 (13)
‘ 1987 (14)
‘ 1986 (15)
‘ 1985 (16)
‘ 1984 (17)
‘ 1983 (18)
‘ 1982 (19)
‘ 1981 (20)
‘ 1980 (21)
‘ 1979 (22)
‘ 1978 (23)
‘ 1977 (24)
‘ 1976 (25)
‘ 1975 (26)
‘ 1974 (27)
‘ 1973 (28)
‘ 1972 (29)
‘ 1971 (30)
‘ 1970 (31)
‘ 1969 (32)
‘ 1968 (33)
‘ 1967 (34)
‘ 1966 (35)
‘ 1965 (36)
‘ 1964 (37)
‘ 1963 (38)
‘ 1962 (39)
‘ 1961 (40)
‘ 1960 (41)
‘ 1959 (42)
‘ 1958 (43)
‘ 1957 (44)
‘ 1956 (45)
‘ 1955 (46)
‘ 1954 (47)
‘ 1953 (48)
‘ 1952 (49)
‘ 1951 (50)
‘ 1950 (51)
‘ 1949 (52)
‘ 1948 (53)
‘ 1947 (54)
‘ 1946 (55)
‘ 1945 (56)
‘ 1944 (57)
‘ 1943 (58)
‘ 1942 (59)
‘ 1941 (60)
‘ 1940 (61)
‘ 1939 (62)
‘ 1938 (63)
‘ 1937 (64)
‘ 1936 (65)
‘ 1935 (66)
‘ 1934 (67)
‘ 1933 (68)
‘ 1932 (69)
‘ 1931 (70)
‘ 1930 (71)
‘ 1929 (72)
‘ 1928 (73)
‘ 1927 (74)
‘ 1926 (75)
‘ 1925 (76)
‘ 1924 (77)
‘ 1923 (78)
‘ 1922 (79)
‘ 1921 (80)
‘ 1920 (81)
‘ 1919 (82)
‘ 1918 (83)
‘ 1917 (84)
‘ 1916 (85)
‘ 1915 (86)
‘ 1914 (87)
‘ 1913 (88)
‘ 1912 (89)
‘ 1911 (90)
‘ 1910 (91)
‘ 1909 (92)
‘ 1908 (93)
‘ 1907 (94)
‘ 1906 (95)
‘ 1905 (96)
‘ 1904 (97)
‘ 1903 (98)
‘ 1902 (99)
‘ 1901 (100)
‘ 1900 (101)
Q30 What do you think makes a good multimedia lesson?
Q31 Do you have any formal training in multimedia learning development?
‘ Yes (1)
‘ No (2)
Answer If Do you have any formal training in multimedia learning de… Yes Is Selected
Q34 What kind of training do you have in multimedia learning development?
Q35 What is your current job?
Q37 Thank you for taking time to complete the survey. Your responses have been recorded. If you have any questions, please contact Dominick Bill Cupuro at dominickcupuro2014@u.northwestern.edu.
‘
Appendix 2 Consent Form
Bill Cupuro Consent Form for study
Note: this is the first question in the survey. If participants answer ‘No’ they are taken to the end of the survey.
Thank you for participating in this research study! This study is being conducted as part of the requirements for the Masters of Science in Learning and Organizational change at Northwestern University.
The survey will take about 15 minutes to complete. Your responses will be anonymous and confidential.
If you have any questions, the researcher’s contact information will be included at the end of the survey.
Do you agree to participate?
Yes
No
‘
Appendix 3 Subject Recruiting Materials
Recruitment e-mail given to key contacts in LinkedIn, Facebook, and key professional contacts.
Appendix 4 Data Collection Timeline
Week 1 Week2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10
Date 9.23-9.29 9.30-10.6 10.7-10.13 10.14-10.20 10.21-10.27 10.28-11.3 11.4-11.10
11.11-11.17 11.18-11.24 11.25-12.1
Assuming IRB approval by October 28th
Proposal Study submitted to IRB 10.20.13 Check eIRB/Submit changes Follow up with IRB (if applicable)
Check for advisor submission (completed 10.20.13) Adjust project timeline if necessary
Survey Create Survey Test
Survey
Create recruitment e-mail Recruit participants via Linkedin, Twitter, and professional contacts
Data Mgmt Review feedback for Survey Check participant demographics Review incoming data Review incoming data Review incoming data Prepare data for analysis