The Testing Manager plug-in is an easy method of assessing your Students on-line using custom-built tests. An unlimited number of tests can be created (and even taken simultaneously) consisting of any number of questions of different types. Multiple choice, multi-part exact answers, and essay-based questions can be designed. Results can be viewed (and published) by the Administrator. Marking is automated (except for essay based answers) and results can be manually adjusted if required. A wide range of options can be configured for each individual test including: time limits, allowing Students to re-take questions (or not), automatically giving Students the correct answer to a question (including an explanation if required) after they submit an answer. Report tables can be generated easily allowing you to summarise Student performance and overall test results.
2. Creating and Managing Tests
The plug-in allows you to create an unlimited number of tests, which are managed individually. There is no restriction on how you choose to deploy your tests. For instance, you may want to create all your tests at the beginning of your course and then periodically enable and disable each test to Students as required.
2.1 Creating a New Test
Upon initially entering the administration page for the plug-in you will be presented with a form for creating new tests as illustrated in fig. 1.
A new test can be created by entering the various fields in this form, as described below:
Once you have filled out the form, click ‘Create New Test’ and a new test entry will appear in a table below as illustrated in fig. 2.
- Test Name. Keep the title short but informative.
- Description. Extra important information about the test (optional).
- Instructions. Specific instructions relating to the test. This will be shown before Students enter the test (optional).
- Time Limit. Enter a time limit in minutes (up to 99999 minutes – integers only). Students will then be timed from when they first enter the test (not from when they first start answering questions). When they reach the time limit, they will be unable to view or answer any more questions. Time limits cannot be used for a test, which allows anonymous access (see below).
- Testing Options. These options relate to the answering of questions within the test:
- Test can be taken without logging in (results not recorded). This option is useful for non-assessed test (e.g. Person self-assessed testing).
- Students can view their own results. Results will be immediately revealed upon answering a question.
- Students can re-take the test. Users will be able to re-take all questions an unlimited number times.
- Students can view the correct answer / explanation (after answering). This option is useful for self-assessed tests. Model answers will be immediately revealed upon answering a question.
- Publishing Options. These options relate to the displayed results of the test when in ‘Published’ status:
- Show complete list of Student scores. A summary of all the results gathered for the test will be shown in a list in order of score.
- Show model answer & submitted answer. Each Student will be able to see each complete question together with their answer and the model answer (including and working / explanation if available).
You may now populate the empty test with questions by clicking ‘Manage Questions’ next to its entry in the table. Please refer to Section 3 for more information.
2.2 Modifying and Deleting Tests
Each existing test can be modified at any point by clicking ‘Modify Test’ next to its entry. This will display the same form as described in Section 2.1 to allow any attributes of the test to be altered.
Clicking ‘Delete Test’ will remove the entire test including any associated question and results gathered.
Both modification and deletion of tests can be performed at any point regardless of the current status of the test and any results that have been gathered from Students.
2.3 Test Status
Each test can be set to one of three status modes as required. These are:
To change the status of a test, select the desired entry from the corresponding drop-down menu. The status will then automatically update.
- Disabled. Test will be disabled for Students. It will not appear on the main website and any attempts to answer questions within it will be rejected. This is the default mode for newly created tests.
- Enabled. Test will be visible and will allow users to answer questions within it. The test will only be visible if it contains one or more questions.
- Published. Test will be visible and will only display results gathered for the test (depending on the publishing options selected). Users will be unable to view or answer questions contained within the test. This mode is used after the test has finished.
3. Managing Questions
Each test can contain an unlimited number of questions. Each question can be one of three different types:
- Multiple Choice Question. Up to eight different choices (A-H) can be used. Only one choice can be elected as the correct answer. Only one score per question (not per choice) can be used. Individual choice cannot be scored differently. (i.e. for ‘half right’ choices). This type of question is marked automatically upon submission.
- Multi-Part Exact Entry Question. This type of question can comprise up to six different answers. A Student must enter an answer for each part which must exactly match the correct answer. This is most useful for short (1-2 words) and numerical answers which contain no ambiguity. Each part is individually scored. This type of question is marked automatically upon submission.
- Essay Based Question. One short essay based answer per question that has a user-defined word limit and score. This type of question must be marked manually by the administrator.
3.1 Creating Questions
Click ‘Manage Questions’ next to the appropriate test entry to reveal the question manager, which allows you to create and manage the question contained within each test. You will be presented with a form as shown below to create new questions as illustrated in fig. 3.
A new question can be created by entering the various fields in this form, as described below:
Only one of the three types of answer listed above can be used for each question. A default score of 1 will be automatically entered for any question with the score omitted.
- Question. Enter your question here. Tags can be used to enhance the text you enter here.
- Multiple Choice Answer. Enter a possible answer for each choice you require (up to H) and check the entry which is the correct answer. Enter an integer for the score.
- Multi-Part Exact Entry Answer. For each part of the answer you require (between 1 and 6 parts) you must enter: The ‘Label’ (describing what should be entered for that part to the Student); The ‘Correct Answer’ (case insensitive); The score (as an integer). The plug-in can ‘intelligently’ mark submitted answers depending on the syntax used for the correct answer. Please refer to Section 3.2 for more information on how to design the correct answer to a question.
- Essay Based Answer. Simply enter the desired word limit for the answer and the total score (as an integer).
- Explanation / Workings / Model Answer. This is an optional field used if further explanation for an answer is required. This is useful for a Student to see the reasons or workings behind the correct answer. Tags can also be used to enhance the text entered here.
Once you have filled out the form, click ‘Create New Question’ and a new question will appear at the top of the table below the form.
3.2 Designing Correct Answers
‘Exact Answer Entry’ type questions are automatically marked by the plug-in. In many cases, a single correct answer may be too ‘exact’ to be of any practical use for automatic marking. There may also be cases where more than one possible answer could be deemed as correct. For example, a question with the answer ‘20%’ could be submitted by Students in a number of different (but still correct) ways. 0.2, .2, 20, 20% might all be considered correct. There may also be cases where a certain margin for error in a (numerical) answer is allowed (e.g. anything between 19.5% and 20.5%). In order for the plug-in to deal with differences and mark an answer accordingly, a number of syntax rules can be followed to achieve an acceptable level of ‘ability’ in the marking algorithm:
An unlimited number of possible answers to a question can be separated by semi-colons:
E.g. red; blue; green; yellow
Any of these possibilities would be marked as correct.
If an answer is completely numerical you can use the ‘[n]’ tag to specify this explicitly:
If the answer is 20%, you would want to allow ‘20’ as well as ‘20%’ (with the ‘%’ sign on the end) to be correct. When the [n] tag is used for a possible answer, the answers submitted will be treated as numerical and any non-numeric characters will be stripped so the answer can be properly equated. Negative and decimal numbers (e.g. -0.25) can also be specified.
If a certain margin for error is required you can specify a numerical range that is deemed correct.
E.g. Any answer between ‘0.8’ and ‘1.2’ (inclusive) is to be marked as correct would be written as:
The low number must precede the high number in the range. Negative and decimal values can also be used (remember that -6 is lower than -5 so would be written as [n]-6~-5) Only one lowest and highest number can be specified for a range.
By default, only the first possible answer specified is shown to students (wherever the model answers have been allowed to be viewed). If you require more than one of the possibilities to be displayed you can use the ‘[s]’ tag to indicate an answer that is to be viewable.
E.g. red; green; [s]blue; yellow; [s]purple
Would be displayed to the students as:
red or blue or purple
You could use the first possibility as the answer (and be more descriptive as it doesn’t have to match any submitted answer) that is publicly viewable and use other possibilities for the actual marking algorithm.
3.3 Modifying and Deleting Questions
Any aspect of an existing question can be modified by clicking ‘modify’ next to the corresponding entry in the question manager. A form as illustrated in fig. 3 will be displayed to allow the modification of any of the attributes of a question as described in Section 3.1. Deletion of individual questions can also be achieved via clicking ‘delete’ next to the corresponding entry.
Generally it is a good idea to make all modifications to the set of questions whilst the test is in ‘Disabled’ mode as any alterations are forbidden as soon as any answer submissions have been made. The only way to make alterations after Students have begun taking the test is by deleting all submissions first via the results manager.
3.4 Designing Questions
The three types of question available have been designed to cover a wide range of typical questions used. Many questions that do not first appear to fit any of these types can be easily altered to sufficiently suit the system.
4. Answering Questions
When user’s visit the Zone containing the Testing Manager plug-in, they will be presented with a list of hyperlinks to the various tests that are currently enabled (or published) and contain questions as illustrated in fig. 4.
If a Test has further instructions or if a time limit has been set, users will be directed to an entry splash screen displaying the extra information as shown in fig. 5. Once ‘Enter Test’ is clicked on this screen, the user will be taken to the start of the test and the timer will start. This is to ensure that the timer is started upon entry to the test and not upon answering the first question (i.e. corresponding to turning over the paper in a conventional test).
Once the Test has started users are required to answer each question displayed using the form provided. An index list of each question available in the test is displayed in the right hand column, which the user can use to manually navigate through the test. Questions in this list that have already been answered are shaded.
Once the user clicks ‘Submit Answer’ they will be taken to a ‘submission successful’ splash screen (as long as a complete answer has been submitted), which will display confirmation of submission and any extra information that has been enabled for the test (e.g. The result and score for the question, model answer and explanation / workings). For tests where results are not being recorded, it is usually a good idea to allow users to see their own results and model answers as they proceed through the test (i.e. for self assessment). If the test is incomplete a link to the next unanswered question will be displayed until all the questions have been answered at which point the user will be notified on their completion of the test.
Time limits are enforced for each user individually. When a new user enters a timed test, the timer will immediately start to count down until the time limit exceeds where the test becomes disabled and no more answers may be submitted from that particular user.
5. Managing Results
To view and manage the results collected for a test at any time, click ‘Manage Results’ next to the desired test entry. A results table will then be displayed showing all the results so far for each Student (the rows) and each question (the columns). Averages for each Student and each question are also shown. The submitted answered together with the current score awarded are displayed in each cell. Purple shaded cells depict questions that need to be marked manually and have not yet received a score.
The Students can either be sorted by when they first started the test or by highest or lowest score.
5.1 Marking / Altering Results.
To adjust the score for any question for each Student, click ‘modify’ next to the required results entry. The submitted answer together with its current score is displayed for each question answered by the Student and can be altered. The score entered must be a whole number that is between 0 and the highest allowable score for the question. The only exception is that essay based scores may be left blank (un-marked). Once the desired adjustments to the scores have been made, click ‘Update Results’ to finalize the changes. This is the only way to view and mark essay based answers.
6. Publishing Results
Once a test is over and all results have been obtained and finalised you have the option of publishing these results on the website for your Students to view. In order to publish a set of results you must change the status of the selected test to ‘Published’ mode as described in Section 2.3. Once changed, the results will be immediately published on the website for Students to view. As described in Section 2.1, there are two main publishing options available that affect the way results are displayed to the Students on the website.
Enabling the ‘Show complete list of Student scores’ option will display a summary list of all the results gathered for the test in order of score as illustrated in the example below. If disabled, the Student will only be able to view their own personal test result statistics as illustrated in fig. 9.
Each logged in Student will see their own name in bold, which can be clicked on to reveal their complete score breakdown including their test statistics as illustrated below.
In addition to these basic results statistics, enabling the ‘Show model answer & submitted answer’ option for the test will allow the Student to see each complete question in the test together with their answer and the model answer (including and working / explanation if available).
7. Report Generating
Results collected for tests can be summarized by clicking ‘Report Generator’ from the main admin screen. This gives you the option of outputting result reports for particular Students or tests, or a combination of the two. From the form at the top of the screen, select the particular test (individual or all tests) and the particular Student (individual or all Students) and click ‘Generate Report’. A table will be generated containing all the information you requested together with averages and percentages for each Student and each test selected. Students that have not taken a test (i.e. gained no mark for a test NOT a mark of zero), are not included in the average percentage for that test. Similarly, only the tests that a Student has taken are included in their average percentage.