Page 8: of Maritime Reporter Magazine (October 2019)

I

INSIGHTS: TRAINING TIPS FOR SHIPS

Murray Goldberg is CEO of Marine Learning Systems which provides software and services to optimize knowledge, skills and behavior in maritime operators. In his former life he was a computer science faculty member at the University of BC researching online learning and assessment delivery models and their effectiveness. This led to him develop WebCT, a learning management system that was used by 14 million students in 80 countries.

Training Tips for Ships

Tip #5 – The Simple Secret to Making

Randomized Exams Fair n last month’s Training Tips for

Ships, we made the important point that we must never give dif-

I ferent people the same exam. If we use an exam over and over, our trainees will very quickly learn what questions are on the exam and share the answers with their friends. Suddenly exam scores begin going up, and time spent learning goes down – both for the wrong reasons. There are few better ways to destroy a training program. he remedy is to always provide different exams. This would be a lot of work, were it not for

Credits: Photo by J. Kelly Brito on Unsplash the exam randomization function found in most web-based Learning Manage- In your LMS, each question category one category with questions all covering ment Systems (LMSs). Randomization should be set up to contain a pool of ? re? ghting PPE at a basic level, one cov- in an LMS ensures that no two people questions which each cover the same ering the same PPE at an advanced lev- get the same exam. But as we wrote last competency at the same level of dif- el, one covering welding PPE at a basic month, this may raise the concern that ? culty. Once we have this set up, the level, one at an advanced level, and so

M TID 2019 R AR EPORT we lose consistency in our assessment exam can be de? ned to always select a on. Now, we con? gure the LMS so that nd practices. After all, it is critical that we speci? c number of questions from each each time it generates an exam, it draws

The 2 Annual report from measure everyone by the same standard. category to make all instances of the (for example) 5 welding PPE ques- the historic MarTID survey

Changing the exams creates inconsis- exam consistent. tions at a basic level, 3 at an advanced initiative to study global tency, correct? Well, no – not if it is level, 4 ? re? ghting PPE questions at a maritime training practic- done correctly. Here’s how. A Simple Example basic level, and so on – depending on es, investment and thought

Let’s consider a PPE exam. For sim- the emphasis we want to place on the

Building Consistent plicity, let’s say the exam covers three exam topics and how dif? cult we want is now available. The sur-

Randomized Exams competencies or knowledge: the pro- the exam to be. From this point on, each veys draw on insights from

The two primary exam variables that tective equipment needed for ? re? ght- time the LMS delivers our PPE exam, it shipowners and operators, we want to deliver consistently are 1) ing, the equipment needed for welding, will contain a different set of questions, maritime education and the dif? culty of the questions, and 2) and knowledge of how to don basic but will always have the same number training institutions and the competencies or knowledge covered protective equipment (clearly this is of questions for each tested competency in the questions. Thus, even when ex- an incomplete exam, but is useful to il- and be of roughly the same overall dif? - seafarers. The insightful ams differ, we want to ensure that they lustrate). Let’s also say that we want to culty. Now we have randomized exams report is available free for all cover the same material to the same ask questions at two levels of dif? culty to help solve the cheating problem but global distribution @ degree, at the same level of dif? culty. to separate those who understand at a have not sacri? ced fair and consistent

In an LMS, all questions are organized basic level vs. those who have mastery assessment.

http://digitalmagazines.marinelink.

into “question categories”. The key to of the subjects.Per our rules above, we Thanks for reading! If you have any com/NWM/Others/MarTID2019a/ html5forpc.html ensuring consistency across these two will now create six question categories questions about this or other training variables in your exams is careful or- – each covering one competency at one tips – send an email to me at: ganization of your question categories. level of knowledge. Thus we will have [email protected] 8 Maritime Reporter & Engineering News • OCTOBER 2019

MR #10 (1-9).indd 8 10/4/2019 9:50:11 AM