Page 8: of Maritime Reporter Magazine (September 2019)

I

INSIGHTS: TRAINING TIPS FOR SHIPS

Murray Goldberg is CEO of Marine Learning Systems which provides software and services to optimize knowledge, skills and behavior in maritime operators. In his former life he was a computer science faculty member at the University of BC researching online learning and assessment delivery models and their effectiveness. This led to him develop WebCT, a learning management system that was used by 14 million students in 80 countries.

Training Tips for Ships

Tip #4 – Never Give the Same Exam Twice e’ve all heard the stories, and many of us have seen it ? rst-hand: the

W completely ineffective assessment practice of giving the same multiple-choice exam to people over and over. The only time this is ever ac- ceptable is when all of the people writ- ing the exam are doing so at the same time and are supervised to ensure they are not sharing answers. However, giv- ing the same exam to different people over a period of months or years ensures that exam grades are going to go up over

Image Courtesy Marine Learning Systems time – sometimes quite markedly. Sad- ly, this is not because people are learn- twice. If you are doing so, now is a great support exam question randomization. ing more. It is because the people who time to stop. So if you have access to an LMS, there wrote the exam earlier are sharing the Does This Mean Constantly Creating is a good chance that you already have answers with those writing it later. This New Exams? the capability.

is no surprise, and it creates real issues. No – though it may mean some chang- What Does This Mean for Assessment

M TID 2019 R AR EPORT

The most obvious problem with the es. Most people who are reusing exams Consistency?

nd

The 2 Annual report from practice of exam reuse is that the assess- are doing so because they are delivering The idea of giving each trainee a dif- ment results become meaningless. They paper-based exams or using poorly built ferent exam may raise the concern that the historic MarTID survey are not an accurate re? ection of the computer-based training. In the case of we lose consistency in our assessment initiative to study global trainee’s knowledge, and they cannot paper exams, it is dif? cult and expen- practices. After all, it is critical that we maritime training practic- be used as a measure of training qual- sive to create a new exam each time it is measure everyone by the same standard. es, investment and thought ity. And, of course, with exam answers given. In the case of poorly constructed However, making it easy to share an- is now available. The sur- in hand, trainees will bypass the learn- computer-based training, the program swers by reusing exams completely dis- ing materials and go straight to the exam is “static”, so the same exam is deliv- torts the playing ? eld – so we already veys draw on insights from – rendering even well-designed training ered to every person who completes the have a problem. Additionally, exam shipowners and operators, useless. training. randomization does not have to be quite maritime education and

Possibly the most damaging effect, The alternative is to randomize your as “random” as it sounds. Instead, done training institutions and however, is that the training program exams. With exam question randomiza- correctly, controls can be applied to seafarers. The insightful loses credibility and respect in the eyes tion, no two trainees will get the same ensure that the same competencies are of trainees. To a trainee, reuse of exams exam. Instead, they will each get a dif- covered, to the same degree, with ques- report is available free for signals laziness and a lack professional- ferent mix of exam questions, and in a tions of roughly equal dif? culty. How global distribution @ ism in the training organization. They different order. This kind of exam ran- do we do this?

http://digitalmagazines.marinelink.

will respond in kind. domization requires a learning manage- Check back here next month and we com/NWM/Others/MarTID2019a/

Thus, the lesson here can hardly be ment system (LMS). But fortunately, will see in Training Tips for Ships! Until html5forpc.html overstated: never give the same exam nearly all modern professional LMSs then, safe sailing!

8 Maritime Reporter & Engineering News • SEPTEMBER 2019

MR #9 (1-9).indd 8 MR #9 (1-9).indd 8 9/11/2019 9:11:25 AM9/11/2019 9:11:25 AM