Executive Summary
This report explores the extension of the company’s training provision to a VLE using a test Moodle installation. Within this Moodle installation, ideas for delivery and assessment have been tested before full deployment. These ideas extend to formats, techniques and pedagogies. Documentation has been created based on the platform and the ideas implemented into it. This documentation has been thoroughly researched and evaluated before its deployment.
For the purpose of testing, a list of fabricated users has been implemented into the installation along with example questions and assignments. Every effort has been made to ensure this data appears credible but only exists for demonstration purposes.
Evidence has been provided of testing in regard to the Moodle appearance, reuse of eLearning material, the capabilities of Moodle and the implementation of a MOOC.
The Moodle appearance has changed in order to demonstrate how Moodle can carry forward the company’s brand image.
eLearning material has been reused in the form of two copyright-free SCORM packages. The advantages and disadvantages have been discussed from this approach.
The capabilities of Moodle have also been explored through the use of forced subscription forums, multiple choice quizzes and assignment upload functionality.
A MOOC has also been implemented featuring validation of peer review tools appropriate to its operation. This has been demonstrated and documented within the report.Table of Contents
1.0 Theming Moodle 5
1.0 SCORM 6
2.1 Advantages 6
2.2 Disadvantages 6
2.0 General Functionality of Moodle: Administration 7
3.1 Forum with Forced Subscription 7
4.0 The MOOC 8
4.0.1 Advantages 8
4.0.2 Disadvantages 8
4.1 Block Diagram of the Peer Review Process 10
4.2 Peer Review & Peer Processes 11
4.2.1 Objectives 11
4.2.2 Key Features 11
4.2.3 Workshop Phases 12
4.2.3.1 Phases Overview 12
4.2.4 Planning a Workshop 13
4.2.4.1 Grading Strategy 13
4.2.5 Define Grades for Submissions and Assessments 13
4.2.6 Allocation Method 14
4.2.7 Instructions and Examples of Work 14
4.2.8 Setup Phase 14
4.2.8.1 Workshop Description 14
4.2.8.2 Submission Instructions 14
4.2.8.3 Assessment Form 14
4.2.8.4 Example Submissions 15
4.2.9 Submission Phase 15
4.2.9.1 Assessment Instructions 15
4.2.9.2 Submission Allocation 16
4.2.10 Assessment Phase 16
4.2.11 Grading Evaluation Phase 16
4.2.11.1 Submission Grade 16
4.2.11.2 Assessment Grade 16
4.2.12 Closed 17
4.3 MOOC Validation 17
4.3.1 Certificates 17
References 19
1.0 Theming Moodle
Figure 1 Theming Moodle
Figure 1 displays the theming capabilities of Moodle. From this example, it is evident that the Moodle installation’s name has been changed to “ThomasMcMurdock”. As well as this, the Moodle installation’s logo has been changed to an image. After changing these visual aspects of the Moodle installation, red hyperlinks were issued to the theme.
These changes to the Moodle installation show how Moodle can be themed to suit the brand of an individual company. 2.0 SCORM
“SCORM actually stands for Shareable Content Object Reference Model. and it’s a technical standard telling authors how their content should be structured so that VLEs can display it without any hassle. The great thing is that the standard also describes how a student’s progress through the activities and quiz results can be published to the VLE. This means that Moodle can remember how far a student has progressed, and the results of any tests built into a SCORM object can be included in the course grade book.” (Wild, 2009)
Figure 2 Windows 10 SCORM package
The reuse of eLearning material has been implemented into the Moodle installation. In Figure 2, one of the two copyright free SCORM packages installed on the SCORM course can be seen.
2.1 Advantages
• Possible to reuse learning material in one common learning environment.
• Standardised so provides a common format for different courses.
2.2 Disadvantages
• Based entirely online and accessed using a web browser.
• API can be easily manipulated using JavaScript.
• Browser standards vary and can display SCORM packages differently.
3.0 General Functionality of Moodle: Administration
3.1 Forum with Forced Subscription
To add a forum with forced subscription in the Sandpit Moodle course, the following steps were taken:
1. Turn editing on
2. Add an activity or resource
3. Select forum
4. Click add button
5. Give the forum a name and/or a description
6. Click subscription and tracking dropdown
7. Select forced subscription from subscription mode
8. Click the save and display button
The implemented forced subscription forum can be seen in Figure 3.
Figure 3 Forum with Forced subscription
3.2 Multiple Choice Quiz
To add a multiple-choice quiz with three example questions in the Sandpit Moodle course, the following steps were taken:
1. Turn editing on
2. Add an activity or resource
3. Select quiz
4. Click add button
5. Give the quiz a name and/or description
6. Click the save and display button
7. Select edit quiz
8. To add a question, click add
9. Click a new question
10. Complete the form regarding the aspects of the question
11. Click the save and display button
The implemented multiple-choice quiz with three example questions can be seen in Figure 4.
Figure 4 Multiple-choice quiz with three example questions
3.3 Assignment Upload Link
To add an assignment upload link in the Sandpit Moodle course, the following steps were taken:
1. Turn editing on
2. Add an activity or resource
3. Select assignment
4. Click add button
5. Give the assignment a name and/or description
6. Attach any relevant files
7. Configure dates and times for submissions from, due date and cut-off date
8. Configure the submission type as file
9. Click the save and display button
The implemented assignment upload link can be seen in figure 5.
Figure 5 Assignment upload link
3.4 Populate Course with Participants
To populate the Sandpit Moodle course with 100 fabricated student names and two teaching roles, the following steps were taken:
1. A CSV file was created in Microsoft Excel containing each of the following details for each user:
a. Password
b. First name
c. Last name
d. Email address
e. Username
2. Turn editing on
3. Click settings
4. Click site administration
5. Click users
6. Click accounts
7. Click upload users
8. Add the CSV file to upload
9. Check the users in the preview are correct
10. Click upload users
11. Check results of upload and exceptions
12. Click continue
13. Return to Sandpit course
14. Click participants from the menu
15. Click enrolled users
16. Click enrol users
17. Select all relevant users and assign roles
18. Click finish enrolling users
The Sandpit course’s total enrolled users and the required examples can be seen in figure 6.
Figure 6 Sandpit enrolled users
4.0 The MOOC
MOOCs, or Massive Online Open Courses, have become popular in the eLearning industry. Below are the advantages and disadvantages of implementing them into the Moodle site.
4.0.1 Advantages
The majority of MOOCs provide free access as the aim is to increase educational awareness and provide free learning materials to anybody with an Internet connection. This explains the word “Open” in the abbreviation. For the most part, MOOC alternatives are expensive as developers and course material writers are required to produce unique course material and software solutions.
It is increasingly common for large institutions such as educational establishments and international businesses to provide their education materials through the use of MOOCs. Most topics can be found to have been implemented into a MOOC.
While using conventional training methods, employers usually give staff a fixed time away from employment or their duties to complete their course. Examples of this extend to security awareness events and off-site employee training. MOOCs provide a flexible alternative and allow for staff to train on site or at home. In turn, staff are available and training can be fit into their daily duties.
4.0.2 Disadvantages
In most cases, MOOCs are not accredited meaning they lack value in terms of professionality. Employers are generally more interested in accredited qualifications. The reason for the lack of accredited MOOCs is the learning material is often extremely simple and cannot provide the same knowledge as a professional with experience of the subject.
MOOCs are purposefully made simple to appeal to a wider range of people who may be interested in learning from them. This also leads back to the accessibility of the learning materials as they are often aimed at beginners.
Another drawback of MOOC usage is the predisposition of learners to lose interest and vacate their course before they complete it. One of the main reasons for this abandoning of courses is related to why MOOCs exist. Lack of structure provided by the MOOC is also required to give learners flexibility with their educational gain.
Finally, another disadvantage of MOOCs is the absence of personal interaction with a professional who is providing the course. At any one time, an unlimited number of learners can be enrolled onto one course meaning personal support is impossible to provide without an unfair bias. Instead, learners look to online forums for answers to their queries meaning they can be provided with incorrect information and lack of reasoning. Mostly, grading of work is also completed by fellow learners without individual feedback.
To conclude, MOOCs provide easy access to a wide variety of high quality learning resources usually for free, while providing the user with flexibility. However, as is common with most MOOC implementations, the courses lack structure and are typically easy to complete leading to no accreditation. Users of MOOCs require initiative and a lot of effort.
4.1 Block Diagram of the Peer Review Process
4.2 Peer Review & Peer Processes
Moodle Workshop features a peer review tool to allow for learners to assess the work of their peers based on a set criteria by the teacher. Individual learner’s Workshop grades are devised from their own assignment submission and their assessment of the assigned worked of other learners. In this section, an overview of Moodle Workshop is provided, explaining the process of Workshop implementation within a Moodle course.
Workshop is considered to be an advanced tool and is to be used by Moodle administrators and managers with a familiarity of the Assignment feature.
4.2.1 Objectives
The purpose of this section is to provide an insight into:
• Recognising the decisions to be made while planning a Workshop implementation.
• Distinguishing between the different Workshop phases and their purposes.
• Developing a Workshop in the Moodle installation.
• Configuring the created Workshop’s settings.
• Distributing learner-submitted assignments to other learners to be reviewed.
• Calculating grades from the Workshop and providing them to students.
4.2.2 Key Features
Moodle Workshop is similar to Moodle Assignment but contains added features, one of which is peer review. Peer review allows for learners to assess each other’s work. Within a Workshop, learners submit their own assignment in the form of text or attached files. In turn, this means group assignments cannot be supported by Workshop.
Each learner’s submitted assignment is assessed using criteria provided in advance by the teacher. The Workshop feature supports different formats of assessment criteria in addition to multi-criteria assessing. During peer review, each learner is provided with another learner’s submitted assignment to evaluate using the provided marking criteria from the teacher. Workshop manages the distribution of submitted assignments to different learners.
Upon completion of the Workshop, a grade is awarded to each learner based on both their own work’s grade and the grade attained from their assessment of other learners’ assignments. Workshop creates two grades within the course’s Gradebook for each of the before-mentioned items. These grades can then be combined if found to be necessary by the course manager.
If necessary, the teacher can provide the learners with example completed assignments to ensure they are able to provide a fair assessment of the work. This helps learners to understand the assessment criteria and the Workshop peer review process. These same examples can be used as a reference by the learners when completing their own assignments to ensure they are providing the expected work.
When all the submitted assignments have been assessed, the teacher is provided with the capability of publishing certain submitted assignments for the rest of the course participants to review. In contrast, the Assignment function on Moodle only allows for the student who submitted the assignment and the teacher to view the work.
Workshop or Assignment?
The tools provided by the Assignment feature of Moodle are found to be a lot easier to use for administrators and teachers than those found in the Workshop feature. Assignment also supports the submission of group assignments with group grades provided to the students working on that particular assignment. Both the Workshop and Assignment functions support the use of predefined assessment criteria.
Workshop provides students with a peer review feature to assess each other’s submitted assignments. In addition to this, Workshop also supports example assignments for learners to use as a guide for their own work. The assignment feature does not provide the facility for submitted and peer reviewed assignments to be published upon completion that Workshop does.
4.2.3 Workshop Phases
Workshop is made up of several successive sections. As each activity is completed, the teacher activates the next section of the Workshop. Activity progress is displayed on the Workshop’s main page with the active section highlighted. Each section contains the tasks to be completed alongside an indicative icon which displays whether the allocated task has been accomplished or not.
Advancing to the next section is completed by selecting the lightbulb next to the section name. Teachers are able to change the activated section at any point during the assignment completion process. It is also possible to configure the Workshop to automatically advance based on time.
4.2.3.1 Phases Overview
4.2.3.1.1 Setup Phase
During the setup phase, the course teacher develops assessment criteria, instructions and prepares example submissions. Then the Workshop settings are configured.
4.2.3.1.2 Submission Phase
When the submission phase is activated, learners submit their completed assignments which in turn are allocated to other learners on the course for peer review.
4.2.3.1.3 Assessment Phase
During the assessment phase, learners complete peer review on the work assigned to them.
4.2.3.1.4 Grading Evaluation Phase
The grading evaluation phase consists of the calculation of students’ grades.
4.2.3.1.5 Closed
Once the closed phase has been activated, students can view their own grades, the completed peer reviews on their work and any associated feedback.
4.2.4 Planning a Workshop
Before the implementation of a Workshop, it is important to plan its format. The decisions made during the planning stage will affect how the Workshop is built and its outcome. Therefore, it is necessary to consider issues associated with configuring the Workshop before it is implemented. This makes Workshop implementation more effective.
4.2.4.1 Grading Strategy
Grading strategy settings determine the approach to be used by learners when reviewing their allocated peer’s assignment. This strategy also regulates the format in which the grading criteria is provided to learners. There are four grading strategies available.
4.2.4.1.1 Accumulative Grading
When choosing the accumulative grading option, the form for the assessment is split into aspects. Each aspect is awarded a score and a comment by the reviewer and is weighted in relation to other aspects. The total grade awarded at the end of the review is calculated by a mean average of the aspect grades.
4.2.4.1.2 Comments
As with accumulative grading, the assessment form is split into aspects. Each aspect is given a comment by the reviewer but no score. The final grade awarded for the assessed assignment is 100%.
4.2.4.1.3 Number of Errors
The assessment for is comprised of assertions. Each assertion is provided with a yes or no answer by the reviewer. These yes or no answers can be altered by the Workshop manager to true or false and other associated terms. The assertions are weighted in relation to each other. The grade mapping tool then converts the amount of errors into a score.
4.2.4.1.4 Rubric
When using the Rubric assessment method, the form is devised of criteria. Each specific criterion contains a number of levels, for example very good, good, okay, bad, very bad. Reviewers select the level which in turn is provided with a numerical value which is used to award the final grade.
Once the grading strategy has been chosen and the submission phase is reached, it cannot be changed without the manager returning the Workshop to the setup phase. Assessment criteria must be declared clearly to learners ensuring they understand the expectations of the Workshop. An alternative would be to provide example submissions of the work to be completed to the learners.
4.2.5 Define Grades for Submissions and Assessments
From the Workshop settings, it is possible for the manager to establish the maximum scores for the submissions and assessments of the Workshop. The default submission grade percentage is set at 80% and the default assessment grade percentage is set at 20%. Both these values are editable as long as the comments grading strategy has not been implemented. This is because no numerical grades are awarded to students using this grading strategy and each learner is awarded 100%.
4.2.6 Allocation Method
Once the submission phase has been activated, learners’ submissions are allocated to different learners for peer review purposes. The method of allocation is decided by the course manager.
When selecting manual allocation, the teacher chooses who reviews who’s completed work. The second option is random allocation where the teacher selects the number of reviews for each submission and the number of reviews to be completed by each learner. Scheduled allocation is the final option. This option allows for submissions to be randomly distributed once the submission phase has been completed and is defined using a date and time in the Workshop settings.
4.2.7 Instructions and Examples of Work
Before implementing the Workshop, it is also important to provide supportive documentation to the learners. Workshop provides a facility for these documents to be implemented into the Workshop. These documents can include: instructions for the Workshop; instructions for submission; instructions for peer review; example submissions; example peer reviews; and a conclusion of the Workshop.
4.2.8 Setup Phase
While the Workshop is in the setup phase, teachers are able to change its settings, edit the grading strategy and create or modify the forms for peer review. Teachers are also able to revert back to this stage at any point during the Workshop to change its settings or stop learners from editing their submitted work. Students are not able to access the Workshop during the setup phase.
4.2.8.1 Workshop Description
The Workshop description is set by clicking the set the workshop description hyperlink. The teacher then types the description into the description textbox and selects whether the description will be visible on the course page or not. It is then necessary to click the save and display button at the bottom of the page.
4.2.8.2 Submission Instructions
After selecting the provide instructions for submission hyperlink, the Workshop settings page is displayed. In the submission settings section, instructions are to be added by the teacher in the instructions for submission textbox. These instructions are shown to the learners when submitting their work. The save and display button must then be clicked at the bottom of the page.
4.2.8.3 Assessment Form
Depending on the chosen grading strategy, a different assessment form will be used for each.
When choosing accumulative grading, aspects are added to the assessment form for peers to review other learners’ submissions. Each aspect is given a description, score and weight.
If the chosen grading strategy is comments, as with accumulative grading aspects are added to the assessment form but only a description is required.
However, if the selected grading strategy is number of errors assertions are added to the assessment form and a description, yes or no responses such as true and false, the score and weight must be provided. The grade mapping table is then configured to reflect upon the necessary grade given centred around the amount of errors in the work.
Finally, if the selected grading strategy is rubric, criterions are added to the assessment form with a description, level and score. Workshop the requests the desired rubric configuration which is either a list or grid.
Once the assessment form has been configured, the save and close button at the bottom of the page must be clicked.
4.2.8.4 Example Submissions
Example submissions can be used to give learners a guide to follow when completing their own work and for practice peer review. If it is found to be necessary to add example submissions the add example submission hyperlink must be clicked. The example submissions configuration page displays options to add a title and content with a file upload button to use if a file is required for the example submission. Once the required settings have been configured, the save changes button at the bottom of the page must be clicked.
Once the example submission has been provided, it must be reviewed to provide assessment guidance for students. The continue button must be selected. The page which follows allows for the peer review to take place and the necessary assessment form is displayed to be completed. Once the assessment form is completed, click the save and close button.
4.2.9 Submission Phase
The next phase of the Workshop is the submission phase. Students submit their completed work for review at this phase. If deemed to be necessary, start and end dates can be activated to confine learner submissions within a certain timeframe. Submitted work is then distributed amongst learners according to the selected allocation method for peer review. Once all work has been submitted, the Workshop can be advanced to the assessment phase meaning learners can no longer add their completed work.
4.2.9.1 Assessment Instructions
To provide assessment instructions, the provide instructions for assessment hyperlink must be selected. Within the assessment settings page, instructions for assessment are added to the instructions for assessment textbox. These instructions are displayed to learners at the review stage of the Workshop. Students are able to review their own work if the use self-assessment checkbox is selected. The save and display button at the bottom of the page must then be selected.
4.2.9.2 Submission Allocation
In the submission allocation settings, an allocation method must be selected to distribute completed work to learners for peer review. The allocate submissions hyperlink should be selected to access these settings. The page which is displayed contains three tabs containing the three different allocation methods.
Manual allocation allows for specific peers to be selected for learner submission review. Once the submissions have been submitted, the teacher selects certain reviewers for each piece of work. If learners have not submitted work, it cannot be selected for manual allocation of peer review.
Random allocation allows for submitted work to be allocated to peers randomly for review. From this tab, the amount of reviews per submission or reviewer can be selected. Moodle will then distribute the submissions to the allocated peers for review.
Scheduled allocation automatically distributes completed submissions randomly at a predefined time and date set by the teacher. If this option is required, the enable scheduled allocation checkbox must be selected, then the number of reviews per completed submission or per reviewer must be set, followed by the save changes button. The scheduled time and date can be set using the submissions deadline setting in the Workshop.
4.2.10 Assessment Phase
Once the assessment phase has been reached, learners review the allocated work of their peers. To begin the review, the assess button next to the completed submission must be clicked. The student is then provided with the assessment form provided by the teacher at the setup phase.
4.2.11 Grading Evaluation Phase
During the grading evaluation phase, the calculations for the submissions and peer reviews are completed. It is not possible for learners to edit their work once this phase of the Workshop has been reached.
4.2.11.1 Submission Grade
The submission grade is calculated using a mean average of the grades provided by all the submission’s peer reviewers. The value is then rounded to the number of decimal places provided in the Workshop settings. At this stage of the Workshop, it is also possible for the teacher to override the grades with their own review of the submitted work or provide a greater weighted grade to each submission affecting the overall mean average.
4.2.11.2 Assessment Grade
Peer reviews are automatically provided a grade by Workshop. The Moodle installation calculates the mean average of the peer reviews and chooses the closest review to the score to rate as the best peer review. The closest peer review to the mean average is graded 100%, with other peer reviews being graded based on how far away from the best review they are.
It is possible for the teacher to configure the settings of the assessment grade. The more rigorous the comparison between peer reviews, the more similarities there need to be between the peer reviews to obtain a higher grade.
If there are only two peer reviews for a submitted piece of work it is not possible for Moodle to decide which score is considered the best. Therefore, both peer reviews are graded at 100%. The stop this from happening, the teacher must either provide an additional review of the work or increase the weight of one of the peer reviews.
4.2.12 Closed
Upon completion of the Workshop, the manager of the course switches the Workshop to the Closed phase. Final grades are calculated and transferred to the course’s Gradebook. During this time, learners can access their grades, submitted assignment, peer reviews of their work, and published submissions by the course teacher.
4.3 MOOC Validation
4.3.1 Certificates
The value of certificates varies considering the depending on the nature of the learning material and the institution providing it. Evaluation of a learner’s gained knowledge can be automated through quizzes and examinations. This evaluation can be scaled up to knowledge obtained from a MOOC. This adds validity to any certificates gained through an examination or quiz under controlled conditions. However, one of the main purposes of a MOOC is that it is widely available and gaining such certificates would require the participants to travel to a particular examination facility.
Regardless of the number of mechanisms used within Moodle to provide an increased grading accuracy, peer review used in a MOOC only acts as a method of reflection and collaboration between its learners as opposed to certifying the learners for their gained knowledge. It is evident that although MOOCs allow learners to access online learning material from anywhere, they are not given the guidance of a professional instructor who can ensure they understand the fundamentals before moving onto more advanced topics surrounding the material on the MOOC. MOOCs have scaled up the accessibility of learning materials but in this same way, demand for experts on the subjects to provide professional guidance cannot be scaled up.
If certification gained from peer review was accredited and given value in an industry related to the material from the MOOC, training staff who have created the course would need to individually review every peer review to ensure each was graded correctly. The certificates gained from peer review-related activities merely represent attendance of the course. Although the certification of peer review is a weak point of MOOCs, such validation could hold value within an individual organisation as long as each review is checked by instructors.
The size of MOOCs allows for instructors to check up on who has completed which tasks and as long as this fact is communicated to learners, they will gain motivation to get the tasks completed.
There are some aspects of learning which cannot be made available online for various reasons including cost. This makes it necessary to extend the MOOC course by offering training to employees at verified training centres by professionals.
Summary & Conclusions
The implemented Moodle MOOC contains basic elements to allow for a low-cost solution to the deployment of the Health & Safety learning programme. The learner will be able to view the introduction during the first week of their course. Once the second week has been activated by the course manager, they will be able to view the first and second weeks and so on.
Figure 7 MOOC course page
As is visible from Figure 7, a video is provided each week alongside a PDF file containing notes on that week’s topic. From this, the learner must complete a brief 500-word overview of the week’s topic for peer review.
Only making the next topic accessible for learners once they have completed the current one motivates them to progress by completing the necessary work every week. From this course, if employees want to become safety officers for their departments they can apply for the position as long as they have completed the course to a high enough standard. This is further motivation for the students.
As previously discussed in section 4.3, the value of certificates obtained from MOOCs is not regarded highly by employers as a learner is not given a set path to follow. Restricting access until topics are complete is a way of giving a certificate gained from completing this MOOC implementation a higher value to the organisation.
However, this form of certification will require moderation from the company’s Health & Safety representatives. Also, there may be issues with timing as holidays and sick days may be taken during the course of the week a topic is activated which will not be fair on particular learners. This means some employees may be held back during the course until there are enough learners at the same stage to complete the peer reviews.
References
Wild, I. (2009). Moodle 1.9 Math. Birmingham [u.a.]: PACKT Publishing.