By Bob Williamson, Contributing Editor
Effective training results in learning to improve one’s knowledge and performance. Ideally, job-related training should result in learning (i.e., becoming qualified to perform the job as expected), mastery of job skills and knowledge. Training methods have changed dramatically over the decades, including the embracing of online and computer-based models. Faster computers, broadband connections and software capabilities have launched a variety of new, more efficient training approaches. Unfortunately, some job-related learning, such as equipment operation and maintenance, doesn’t lend itself to computer-based training (CBT). The “hands-on” elements of job-performance requirements are lacking with CBT. Furthermore, students with tactile learning styles struggle with the visual and auditory methods common to CBT.
A recently published report on the effectiveness of CBT in the U.S. Navy concluded that maintenance costs have increased “tens, if not hundreds, of millions of dollars” due to the Navy’s implementation of CBT: After this type of training was introduced, system-specific maintenance costs rose, excess maintenance hours accrued due to productivity losses, and additional education was required to achieve initial ratings. (More on this analysis later.) We should learn from the Navy’s experience.
From schoolhouse settings to CBT
Until 2003, the U.S. Navy employed traditional schoolhouse instruction methods. Non-commissioned officers from the Operational Fleet were assigned to shore duty to serve as instructors. Their career experience provided students with important lessons from the fleet, anecdotal insights and case studies. Hands-on experiences in laboratory settings using the exact equipment found aboard ship with instructor-introduced flaws and defects for troubleshooting and repair supplied an important real-world aspect to the training. Lectures and demonstrations could also be adjusted to suit the needs of individual students, if needed.
This traditional approach to instruction was labor-intensive; demanded time away from the fleet by experienced officers; called for a duplicated capital investment for equipment; and appeared to require more training time than emerging computer-based methods—facts that concerned Congress and the Navy brass.
So, between 2003 and 2004, the U.S. Navy shifted “Class A” technical training from traditional schoolhouse settings to computer-based offerings to reduce costs and training time prior to on-ship duty. Subject-matter expert instructors were replaced with learning facilitators. Students moved through learning modules at their own pace on personal computers.
Computer-based training under review
Responding to concerns from the fleet about the knowledge level of sailors and anecdotal information regarding CBT, the Naval Inspector General (IG) initiated a review and assessment of the Navy’s computer-based training program in May 2008. “Computer Based Training” was the subject of the IG’s Report to the Secretary of the Navy in March 2009.
The IG’s study defined CBT as an “individual or group-based instruction using a computer as the primary training medium.” The review found that the courseware content and quality varied widely, delivery systems were outdated, and the instructional design of CBT curricula did not capitalize on adult-learning theory principles. Researchers also found that CBT was most effective in a blended learning environment versus a stand-alone training medium. Other findings included the following:
Job-related training—The study found “no mechanism in place to ensure the course content was linked to sailor work.” There was little subject-matter expert (SME) or instructor input in course content development. This resulted in a major disconnect between the training content and the on-job performance requirements. While the fleet expected an operator who can perform basic maintenance, the training curriculum trained in system operations only. (Another interesting finding related to what was a generally accepted assumption. The report noted that “while a generation of Americans may be proficient in using computers for gaming and [social] networking, most have no experience learning academic or technical material via personal computers.” CBT proved largely ineffective with this generation.)
Technological demands—Computer-based training requires access to personal computers (PC) or electronic classroom connected to network servers and a comprehensive learning-management system (LMS) to electronically monitor progress, grades, and transcripts. Multi-media learning content and multiple users require up-to-date computers and sufficient network bandwidth capacity. The Navy IG study found that outdated computers, insufficient network bandwidth and limited internet availability frustrated students: computers froze, pages loaded slowly or timed out resulting in unplanned course downtime.
Sea stories—One of the instructional elements often missing in a CBT learning environment is the voice of experience. Instructors with real-world experience who can relate actual situations regarding the subject matter and answer specific student questions about it are invaluable parts of the learning process, especially for adults. Students are motivated, learning is reinforced, retention is increased and the ability to transfer learning to situations outside the classroom is enhanced. “Sea stories” help put the subject matter in context of the world of work and make learning enjoyable for both the student and the instructor.
Outcome-based training—The U.S. Government Accountability Office (GAO) also conducted a review that included training effectiveness. Its findings, published in the “Report to Congressional Committees on Military Readiness” (GAO-10-592) in 2010, stated that since 2003 the Navy “has made significant changes to its training programs.” In the move to CBT, however, the GAO cited a lack of outcome-based performance measures to fully evaluate the impact of these training programs, as well as a lack of complete data to track personnel qualification times. The report concluded that “while achieving (training) efficiencies is very important, particularly given the growing costs to compensate personnel, the Navy must still ensure that it can perform its mission safely and maintain adequate readiness.”
Sonar-system maintenance
The IG and GAO reports prompted further empirical study, so a detailed analysis of the Navy’s CBT approach to system maintenance was performed. The findings were published in Applied Economics in September 2014. The study—“Does computer-based training impact maintenance costs and actions? An empirical analysis of the US Navy’s AN/SQQ-89(v) sonar system”—focused on one type of Navy sonar system with a data analysis of maintenance labor and costs, before and after CBT implementation. The report “determined that CBT’s use has adversely influenced the parts costs, actions and labor hours associated with operating and maintaining the AN/SQQ-89(v)” (sonar system). Other findings were as follows:
Cost reductions cost more: While training costs and training times were reduced with deployment of CBT, actual on-the-job performance and proficiency were harmed. Specifically, system parts costs increased by $4,971 per year, maintenance actions increased by 21 to 32 to 36 per year and maintenance labor increased by 730 hours.
The study also revealed that sailors reporting to the fleet are not as well prepared as the classroom-trained sailors of the past. Predicted excess maintenance costs for the sonar system since 2003 were some $16 million higher due to the Navy’s introduction of CBT, according to the report. The estimate did not include lost productivity due to excess maintenance hours and additional education required.
Learning from the Navy’s experience
Maintenance and reliability professionals are experiencing the impact of skills shortages in today’s workplace, and are concerned for the future. Many community colleges and tech schools do not offer the training needed for our equipment and facilities. High schools have dramatically reduced (or eliminated) career education and hands-on technical training programs relating to the skills and knowledge required for industrial and facility installation, maintenance and repair. The temptation to jump on CBT as the fast-track to training our future maintenance staff is huge—and, by itself, hugely ineffective.
Just because members of the younger generation may be proficient at using computers, tablets and smart phones doesn’t mean they will be proficient at maintaining digital generation equipment. Nor does it mean they can learn technical skills from CBT modules. Hands-on abilities, mechanical aptitudes and problem-solving in real-world equipment situations are a must.
Computer-based training is a viable methodology when content is directly applicable to the workplace (i.e., job task analysis), facilitated by skilled practitioners (mentors) and coupled with hands-on experiences on actual equipment (in a lab setting or formal in-plant on-job training). Learning to become proficient at industrial maintenance requires much more than learning from a computer program.
Because of the hands-on nature of the work and the learning style of most people who pursue maintenance careers, hands-on learning must be the primary approach in developing industrial-maintenance technicians. It’s critical that we prepare our technicians of the future with efficient, effective job-related programs that combine instructional media, classroom training, hands-on experiences and on-the-job mentoring by skilled and knowledgeable
practitioners. MT
References:
NAVINSGEN Computer Based Training Study, U.S. Naval Inspector General Report to the Secretary of Navy in a March 2009.
The U.S. Government Accountability Office (GAO), Report to Congressional Committees on Military Readiness (GAO-10-592), June 2010.
Does computer-based training impact maintenance costs and actions? An empirical analysis of the U.S. Navy’s AN/SQQ-89(v) sonar system, Robert M. McNab & Diana I. Angelis; Applied Economics, 46:34, 4256-4266, DOI: 10.1080/00036846.2014.955254).
Bob Williamson, CMRP, CPMM and member of the Institute of Asset Management, is in his fourth decade of focusing on the “people side” of world-class maintenance and reliability in plants and facilities across North America. Contact him at RobertMW2@cs.com.
