The similarity measures are modified by certain weights and thresholds which have to be determined for the data set under test. We then compare the results of the automatic marker to the moderated human marks. We mark diagrams by comparing a student’s attempt with a gold standard – a specimen solution (there could be more than one acceptable solution) – to produce a set of similarity measures for certain diagram features known as minimal meaningful units (MMUs) (Smith et al., 2004) to which the mark scheme used by the human markers is applied. Of course, the user may not be able to fix the errors and can submit the syntactically incorrect diagram to the automatic marker which has to operate in the face of such errors. If a repaired diagram is re-checked, errors may still be detected, but these are the irreparable complex errors which the user should attempt to fix prior to submitting the diagram to the automatic marker. Other errors will be repaired on the basis of the most likely error – there are some patterns which occur frequently and have highly likely causes (see Figure 4). Simple errors, by definition, are those that can be repaired without changing the essential meaning of the diagram. Not all errors are repaired – only those that can be dealt with with reasonable certainty.
Having identified errors, the tool can be asked to repair the errors (see Figure 3). A textual description of an error can be obtained by right-clicking on the erroneous element and requesting feedback as shown in Figure 2. The syntax checker identifies errors by shading (objects and activations) and change of colour (messages) – although this mechanism may change after usability testing. This facility can be disabled if required, for example if the tool is used in an exam. tool shown in Figure 1 incorporates a syntax checking facility (within the Tools menu).
0 kommentar(er)
