CHL
Supervision Continuing Education (SCE)
SELF-STUDY PLANS

Supervisory Continuing Education (SCE) lessons provide members with ongoing education focusing on supervisory or management issues. These lessons are designed for CHL re-certification, but can be of value to any CRCST in a management or supervisory role.

You can use these lessons as an in-service with your staff, or visit www.iahcsmm.org for online grading at a nominal fee ($5 per single lesson plan, or bundled packages are available for quantities of 6 lessons for $25 (save $5) or 12 lessons for $50 (save $10) for greater savings).

Each lesson plan graded online with a passing score of 70% or higher is worth one point (contact hour). You can use these points toward either your re-certification of CRCST (12 points) or CHL (6 points), but you can not use them for both.

Mailed submissions to IAHCSMM will not be graded and will not be granted a point value (paper/pencil grading of the SCE Lesson Plans is not available through IAHCSMM or Purdue University; IAHCSMM accepts only online subscriptions of the SCE Lesson Plans).

Lesson Plan CHL 302
Training Basics for Central Service Managers: Part VIII (Training Evaluation Procedures)
[Reprinted from Communiqué: May/June 2007]

Figure 1 shows how far we have come in our extended discussion of training. We have now reached the final step: evaluation.

Figure 1 notes that the results of the evaluation can suggest the need to repeat the entire planning process from Step 1 when training needs were identified or, alternatively, to refocus efforts on other planning steps. Consider a training program to improve supervisory skills to reduce turnover. Assume that the initial definition of training needs (Step 1) focused on supervisory shortcomings related to interaction with staff members, and that a training program was planned and implemented (Steps 2 – 8) to improve these skills. Also assume that after-training evaluation indicated that the turnover rate was not reduced. Instead, analysis revealed that the problem (Step 1) was really scheduling, compensation or
recruitment/selection of the “best” applicants. In this example, it may be necessary for trainers to plan and implement an entirely different training
program to address turnover. In (hopefully!) a much more common situation, after-training evaluation may indicate that the program was less-than-successful. While it focused on the correct problems (Steps 1 and 2), other activities, such as training plans and/or training lessons, failed to meet expectations. Then only selected steps rather than the entire training process might need revision to improve training results.

Need to Evaluate Training
Why is training assessment necessary? No healthcare facility has access to all resources that would ideally be helpful to attain goals. Time, money and labor are among the resources that are in an increasingly limited supply. Central Service Managers must determine whether their commitment to training is a better use of these resources than alternative uses. This is one reason that the evaluation of training efforts is important. Other reasons may be to:

• Assess the extent to which training achieved planned results – Training objectives have a two-fold purpose: to identify competencies that will be addressed in training and to provide a benchmark against which training can be evaluated. If one objective of training is to teach a Central Service Technician how to decontaminate an endoscope using specified steps in an operating manual, this objective drives the training (how to decontaminate the device). It also provides a way to evaluate training effectiveness (the extent to which each required operating step is sequentially used).

• Identify strengths and weaknesses of training – Few training programs are 100% effective or ineffective. Some training lessons are better than others, some training activities are more useful than their counterparts, and some trainers are more effective at delivering training than their peers. Evaluation identifies aspects of training to continue and to revise.

• Determine success of individual trainees – Trainees who are successful (they achieve planned results on the job) will not require
  additional training. Other staff members may need additional training assistance. Assessment of individual trainees is relatively
  easy when an individualized, on-job training method is used, and it is more difficult when a group training process has been implemented. The assessment is, however, equally important.

• Gather information to help justify future programs – When the success of a training activity is quantified, objective information can help determine whether future training efforts can be justified or whether resources are better invested in other alternatives to improve effectiveness.

• Determine trainees eligible for future training – Some facilities may offer educational or training activities in formalized career development programs that require prerequisite training. Others may have formal or informal “fast-track” programs in which selected trainees who have successfully completed training are eligible for additional training. These, in turn, lead to increased promotional considerations as vacancies occur. In both instances, it is important to know whether and to what extent individual trainees successfully completed the training.

• Assess costs/benefits of training – The expenditure of any resource must generate a return greater than its cost. Some benefits of training including improved morale and increased interest in attaining quality goals are difficult to quantify. Others including improved customer service skills and reduced operating costs for a specified task may be easier to quantify, and both could be assessed with a training evaluation activity.

• Reinforce major points for trainees – Some training evaluation methods such as objective tests and performance appraisal interviews
  allow managers to reinforce important training points during the evaluation. Questions on a written assessment likely address the most important training concepts. If they are self-graded or reviewed by the manager, reinforcement of priority information becomes possible. Likewise, if performance appraisal interviews address training concepts, additional reinforcement opportunities arise.

• Assess trainees’ reactions to training – Managers who are interested in improving training programs want to learn their trainees’ perspectives about the programs. Anonymous input gained before, during and after-training can be helpful in doing so.

• Assess managers’ reaction to training – “There’s always a better way,” is an old saying that applies to training as well as to other
  management tactics. Those who have used a training lesson may have ideas about ways to improve it (and, therefore, the training).

More About Evaluation
Many Central Service Managers consider evaluation in the context of an after-training assessment. While training should be evaluated at its completion (and, perhaps many months after its completion as well), evaluation can also be helpful before training even begins and during its conduct.

A training evaluation method should address several concerns:

• Training evaluation methods must be valid; they must measure what they are supposed to measure. Assume that a training objective focuses on the ability of trainees to successfully complete an inventory count. Assume also that the training assessment only asked trainees about issues such as“Did you like the training?” and “Did the trainer seem enthusiastic?” It is not possible to learn if training objectives were not attained.

• Reliable – Training evaluation methods are reliable when they consistently provide the same results. Training activities implemented
  in the same way by the same trainers using the same training resources and procedures to staff in the same position should have
  similar results.

• Objective – Training evaluation methods are objective when they provide quantitative (measurable) assessments. Acquisition of
  knowledge can be subjectively measured by performance on a well-designed test. Efficiency in a skill might be best be assessed
  by observing the trainee’s acceptable performance of the task after training.

• Practical – A training evaluation method is practical when the time and effort required for the assessment is “worth” its results.
  Knowledge assessments that require trainees to memorize mundane facts, and skill demonstrations that are benchmarked against staff with extensive experience and efficiency in performing the task are not practical.

• Simple – An evaluation method is simple when it is easily applied by the trainer, easily understood by the trainees and when results are easy to assess and analyze by those evaluating the training.

When to Evaluate Training
Assume that a trainee participates in a training session and missed only two questions of 20 on an after-training test. Most managers would likely conclude that the training was successful because the participant scored 90% (18 questions ÷ 20 questions). In fact, the training could have been a waste of the facility’s resources if the trainee already knew the concepts addressed by the 18 questions answered correctly before the session began. In actuality, the after-training evaluation really measured what the trainee knew when the training was completed rather than what he/she learned from it.

To address this concern (in other words, to determine what trainees actually learn from training), some trainers use a pre-test/post-test evaluation. A manager can first identify key concepts to be addressed during training. These are addressed in a pre-test before the training begins. This same measurement tool is used again at the conclusion of the training. The improvement in scores between the pre-and post-test represent a measure of training effectiveness.

In addition to providing an objective measure of the value of training, other advantages of pre-test/post-test evaluation can include:

• It provides trainees with an overview (preview) of the training;
• It helps trainees to identify some of the most important concepts to be addressed in training;
• It allows managers to preview the lesson and suggest priority learning points before the training begins.

As is true with other methods of training evaluation, it may be helpful to assess training several months (or even longer) after the training is completed. Than the manager can determine the extent to which training information has been retained and, more importantly, applied in the workplace.

Effective trainers inform trainees during an introductory session that they will be asked for feedback during the sessions. As this is done, trainers
can obtain a “first-half” reality check to improve the training that has yet to be presented. Managers facilitating group sessions can ask trainees to write anonymous responses to questions such as:

• I wish you would no longer…
• I hope we continue to…
• I don’t understand…
• I hope you will start to…
• The one concept that I wish you would discuss further is…
• The one concept that I really want to learn that has yet to be discussed is…

After-training evaluation can help trainers assess whether training achieved its planned results. It helps to identify how training sessions might be improved and determines the success of the trainees. Experienced trainers often use more than one method and analyze the combined results of different assessments to yield a comprehensive overview of training results.

Training Evaluation Methods
There are several training evaluation methods in common use, and the “best” are those applicable to the specific training objectives. Among the alternatives are:

• Objective tests – These can be written, oral and/or skill-based. Written and oral assessments are typically used to assess after-training
  knowledge, and skill-based assessments address physical (skill) proficiencies required to perform a task. Written exams can be
  multiple-choice, true/false, fill-in-the-blank, matching, short-answer or essay. Objective tests have only one correct answer, little or
  no interpretation is needed, and minimal ime is required for trainees to complete the exam and for trainers to score it. Lengthy exams can be computer-scored sing optical scan [op-scan] sheets that are completed by the trainees.) Objective measurements should be written after- training objectives and instructional materials are developed, and a separate assessment should be used for each performance objective in the training lesson.

• Observation of after-training performance – Managers, supervisors and trainers can manage by “walking around” and, in the process, note whether knowledge and skills taught are being applied. When proper procedures are used, a “Great job!” compliment is in order. By contrast, a coaching activity to remind the staff member about incorrectly performed procedures may be needed.

• Rating scales – A simple rating scale can allow trainees to rate (from very unacceptable to very acceptable) factors such as whether
  training meet expectations, and if it was well-organized. Evaluation forms can also include open-ended questions such as“What were the three most useful (least useful) aspects of the training?”

• Interviews with trainees and/or trainers – The use of oral open-ended questions by trainers, managers, mentors and/or human resources personnel may provide useful input from the trainees’ perspectives. As well, trainers can be questioned by their supervisor and/or human resources staff for the same purpose.

• Trainee surveys (evaluation forms) – Trainees can be questioned about their training perspectives immediately after-training, months after-training and/or during performance evaluation sessions. Also, general “staff opinion” surveys can, in part, address training issues.

• Third-party opinions – Feedback from customers can be helpful to assess training designed to address aspects of products and services that directly affect them.

• Analysis of operating data – Training that addresses customer service and labor costs, for example, should result in increased customer scores and lowered labor costs if it is possible to separate components of these data to determine how they were influenced by training.

• Exit interviews – Formal and even informal conversations with staff members leaving the facility can provide input helpful for training evaluation. Unlike their peers, most departing staff will likely have little or no concerns about providing candid and frank responses to queries about training and other issues.

Follow-Up Documentation
Documentation is a final part of training evaluation. Training records should be maintained in the applicable staff member’s personnel file and should include:

• Name of trainee
• Training dates
• Training topics
• Notes about successful completion
• Other information as applicable

This documentation is useful for planning professional development programs, for consideration of staff member promotions, and for input to interviews for performance evaluation purposes. Documentation of training is also helpful when managers develop long-or short-term plans that address training and professional development opportunities for staff members.