Page 17: of Maritime Reporter Magazine (March 2016)

erage, how long learners are spending on Overall Exam Performance Metrics Self-test Metrics materials and then, later, re-attempt the that page (referred to as “dwell” time). Another metric that is easily generated Most learning management systems self test. This latter pattern will create a

Trends in dwell time can be tracked to by most LMSs is that of overall exam include self-evaluation (or self-tests) as lower average score on the self-tests.

determine whether training time expec- performance. This is a metric which can one of their features. Self-tests are typi- A learning management system will tations are valid, and to alert organiza- be maintained without an LMS, but an cally automatically graded, but no grade provide far deeper insight into the ef? - tions to changes which could be caused LMS does the work automatically and is recorded and the trainee is not evaluat- ciency and effectiveness of training pro- by candidate differences or changes in usually presents tidy and ? exible report- ed on the basis of any self-tests they take. grams than we could ever hope for with- the training materials. ing options. Typically, for each exam in Instead, they are used as a formative tool out the use of an LMS. In some cases,

More importantly, dwell time can be the system, an LMS will report on the by the trainee to determine whether they these metrics can highlight signi? cant used to identify pages which are outliers. average score that trainees have been possess the knowledge required to prog- shortcomings in training which can lead

For example, our experience has shown achieving on that exam. This simple ress to the next learning module or exam. to safety or operational problems. This that dwell times for most learning pages metric allows an organization to “keep Because self-tests are used for a differ- is one of the bene? ts of an LMS which fall into a range of about 30 seconds to their ? ngers on the pulse” of assessment ent purpose than summative assessments has made their use very common outside one minute, but there are a small num- performance. It is a useful audit metric (“? nal” exams which test competency), the maritime industry, and which is now ber of pages with very short dwell times, that can be tracked over time to watch metrics about self-tests tell us something greatly accelerating their use within the and another small group with very long for increases or decreases. Such varia- very different than the metrics discussed maritime industry. dwell times. In either case, the LMS has tions must be examined carefully since above. Summative assessment metrics exposed these outliers, alerting the orga- they could be due to a number of factors tell us whether the trainee eventually nization to the potential training issue. including changes in the level of knowl- learned the material. Self-test metrics,

A quick visit to that page will usually edge of incoming candidates, changes in on the other hand, tell us how quickly make it immediately apparent whether the exams themselves, changes in trainer the trainees learned the material. there is a problem, and if so, how it can preparation and performance, changes The reason for this is due to the typi- be resolved. These metrics provide a in training curriculum, and so on. All cal usage patterns of self-tests. Trainees window of insight into how trainees use of these are important and may require typically ? rst review the learning mate- the learning materials; an insight which attention. Regardless of the cause, they rials, and then when they believe they

The Author would not otherwise be available. This is provide insight which should be tracked. know it reasonably well, they move on

Murray Goldberg is CEO of Marine Learn- a great example of how technology can Numbers which are (or are trending to to the self test to validate their belief. If ing Systems (www.MarineLS.com). An be our friend when analyzing training be) too high or too low in relation to all goes well, they move on to the next eLearning researcher and developer, his activities for the purpose of continuous organizational goals indicate a problem module (or the summative test if they are software has been used by 14 million improvement. with training or assessment and must be ? nished). If, instead, the self-test yields people worldwide.

dealt with. a low score, they return to the learning www.marinelink.com 17

MR #3 (10-17).indd 17 3/2/2016 9:49:18 AM