When an organization delivers training to its teams, it needs to know how effective it has been. Are its people putting their learning into practice? Moreover, is it positively impacting their role and the wider organization? The organization has invested a lot of resources into employee training and development and with that comes an expectation to measure its impact. After all, if it cannot measure it, it can’t improve it. Organizations are all about improvement, so they are looking for a guide that will tell them everything they need to know about how to evaluate a training program. Research shows that when it comes to allocating resources to training, the highest priority among U.S. companies in 2019 was increasing the effectiveness of their training programs (Deller, 2020).
Training Evaluation
A major reason to evaluate training programs is to determine whether the training programs are accomplishing their specific training objectives. The process of training evaluation has been defined by A. C. Hamblin as “any attempt to obtain information on the effects of training performance and to assess the value of training in the light of that information” (Deller, 2020).
Training evaluation can be defined as any attempt to obtain information (feedback) on the effects of a training program and to assess the value of training in the light of that information for improving further training. Evaluation of training can be viewed as a method of measuring the change in knowledge, skills, attitudes, job performance, costs, and the quality of the training facilities. A major reason to evaluate training programs is to determine whether the training programs are accomplishing their specific training objectives.
A training program that does not change employees’ knowledge, skills, or attitudes in the desired direction should be modified or replaced(Santana, 2019). A second reason for training evaluation is to assure that any change in the trainees’ capabilities is due to the training program and not to other conditions. To determine that a training program is responsible for changes in trainees, it is necessary to compare the trainees’ performance before and after the program with a control group. Evaluation of a training program becomes necessary to find out how far the training program has been able to achieve its aims and objectives. Such an evaluation provides useful information about the effectiveness of training and the design of future training programs (Santana, 2019).
Such an evaluation also guides the management to update or modify its future training programs. It also enables the management to collect useful data about the success or otherwise of the training program and based on such data, the management can judge the relevance of training and its integration with other functions of personnel management.
Training effectiveness refers to the degree to which trainees can learn and apply the knowledge and skills which they have acquired in the training program. The effectiveness of training depends upon such factors as the attitudes, interests, ability, willingness, values, and expectations of the trainees and the training environment(Santana, 2019).
The training program will be more effective if the trainees are eager to learn, if they are involved in their jobs and if they have career strategies. Further, the contents of the training program, teaching methods, trainers’ ability to teach well, and their ability to motivate also determine the effectiveness of training (Deller, 2020).
Training Evaluation Questions
Advertisment
The following ten sample questions should provide a helpful example of the types of questions that can be included in a participant training evaluation response survey:
Question #1. Did the training content meet your expectations?
Question #2. Was the size of your training group appropriate?
Question #3. How would you rate the quality of the training?
Question #4. Was the mix of presentations and activities suitable?
Question #5. How would you rate the quality of the instructor?
Question #6. Did you learn anything new?
Question #7. Was the training relevant to your needs?
Question #8. Was the course practical and/or easy to apply?
Question #9. Would participants recommend the training to colleagues?
Question #10. Do you have any suggestions to improve this course?
How to improve the user-friendliness of your survey?
While the particular questions you ask will depend on the stakeholders’ expectations, the goals of the training, and the goals of your evaluation, there are certain techniques you can use to improve the quality and usefulness of your survey.
- Balance the number of questions
Keep in mind that the fewer questions you have the higher completion rate you will likely have as well. However, more questions are more likely to render high quality in the reporting later on. Make sure to find the right balance.
- Keep the question short
Keep the questions brief and clear. Avoid using abbreviated words or jargon.
- Put the questions in a logical order
Structure your questions so that those requiring a simple ‘Yes’ or ‘No’ response come first and more open-ended responses come later.
- Keep each question separate
Avoid ‘branching’ questions where one question is dependent on the response to the previous question. Branching will lead to confusion and lower the response and/or completion rate of your survey. Separate each question to make it as clear as possible.
- Conduct a pilot test
Before deploying your survey, it’s a good idea to test them on a small sample group. Share the questions with stakeholders to ensure they are linked with their expectations. Conduct a pilot test with people from your target audience. You may contain an additional open-ended question such as ‘Please let us know of any difficulties or complications in completing this survey’. This can give you valuable feedback as to which questions need rewording or revising.
Training Evaluation Forms
A training evaluation form is a tool used by training managers and human resource professionals to collect feedback from trainers and trainees. It is used to identify skill gaps and problems to help improve training programs and the overall experience these programs provide (Safety Culture, 2020). Below is a snippet of what a training evaluation form looks like:
Image source: Google images
Training Evaluation – Levels: Pre-Training Evaluation, Intermediate Training Evaluation, and Post-Training Evaluation
Advertisment
The training segment has to evolve criteria for evaluating the impact of training on employees (Santana, 2019). Generally, four different criteria are used to evaluate training programs namely the reaction of trainees, knowledge acquired, behavior modification, and other job performance parameters like reduced accidents, increased productivity, lowered absenteeism leaping sales, etc.
Pre-Training Evaluation:
In this stage, an evaluation is made at the beginning of the training program to understand the expectations of the trainees from the training programs and the extent to which they have understood its objectives (Santana, 2019). This step enables the training segment to modify the training curricula in such a way that the objectives of the training program are aligned to those of the trainees.
Intermediate Training Evaluation:
The training and development segment wants to ensure that training is progressing as expected. Mid-course corrections can be made in the event of deviation from the envisaged objectives. For example, if trainees perceive that a training program is aimed at building communication skill is more theory-oriented, rather than practice-oriented, the feedback may be useful to modify the instruction method (Santana, 2019). Thus, it serves as a verifying tool.
Post-Training Evaluation:
The criteria used for assessing the impact of the training program include Reaction, Learning, Behaviour, and Results (RLBR).
- Reactions: This measures the degree of satisfaction of trainees with the training program, namely subject matter and content of the training program, the environment, methods of training, etc (Santana, 2019). The outcome of the evaluation of reaction may be useful in further strengthening the areas the participants find it more useful and in modifying the areas they find it not useful. Negative reactions may dampen the spirit of participation in future training programs (Santana, 2019). However, positive reactions may not provide complete information about the effectiveness of the program.
- Learning: It measures the degree to which trainees have acquired new knowledge, skill, or competencies. The trainer has to measure the knowledge and skill level of trainees at the beginning of the program. It is supposed to be the baseline or standard (Santana, 2019). Again the level of knowledge and skills obtained at the end of training is measured and compared against the standard. Thus pre and post-training comparison helps to assess the improvement level (Santana, 2019).
- Behavior: Similarly, a comparison of pre and post-training behavior may reveal the impact of training on behavior modification (Santana, 2019). Yet, unfortunately, much of what is learned during training cannot be used on the job owing to a lack of resources or a conducive environment. In such a case, one cannot say that the training is ineffective. It follows that when the training environment is similar to the actual work environment, such a climate facilitates the transfer and application of learning (Santana, 2019).
- Results: Generally, it is difficult to measure precisely the impact of training on business performance which depends on several other factors like economic climate, marketing, size of the investment, etc. However, certain measures like productivity, sales volume, and profit, etc., may be compared before and after the training episode. Any improvement may be partially attributed to the training imparted. Besides, return on investment, cost-benefit analysis, and bench-marking are other methods to assess the value of training (Santana, 2019).
Training Evaluation – Purposes and Principle
Certain purposes guide and dictate the need for evaluation; the purposes behind training evaluation are wide enough (Santana, 2019). Training evaluation will have at least one purpose as its primary focus. If the training cannot fulfill achieving the planned business need, then you are required to identify the root cause and take remedial measures (Santana, 2019).
The purposes of the training evaluation are as follows:
1. To justify the role of training, considering budget availability and cutback situations
2. To improve the quality of training for employee development, training delivery, trainer deployment, duration, methodology, etc.
3. To assess the effectiveness of the overall program, quality, and competency of the trainer
4. To justify the course through cost-benefit analysis and ROI approach
The evaluation data, once collected, takes many forms, and is highly valuable.
It can also be used to do the following:
1. Provide feedback on whether the training or development activity is effective in achieving its aims
2. Indicate the extent to which trainees apply what they have learned back in the workplace (transfer of training), an issue which many organizations find they have problems with
3. Provide information on how to increase the effectiveness of current or later development activities
4. Demonstrate the overall value and worth of development activities.
The purposes of evaluation further extend to gauging the effectiveness, effects of training on shaping attitudes, improving performance, reducing rejections, lowering machine downtime, enhancing job quality, enhancing the market share, penetrating new markets, increasing sales, improving quality of work-life, promoting interpersonal communication, etc (Deller, 2020).
The preceding factors speak about the complexity of any effort to evaluate training. These factors further emphasize the importance of being clear about the purpose and the process of evaluation.
An effective evaluation must be carefully planned while designing the training. Then, the evaluation plan should precede training and not follow it (Deller, 2020). Meticulously planned and well-conducted evaluation provides useful information to the institute, the trainer, participants, and sponsoring organizations.
The output of the training evaluation will serve:
1. To illustrate the real worth of a training
2. To pinpoint where improvement is required in forthcoming training programs
3. To assess the effectiveness of the overall course, trainer, and the training methods
4. To carry out a cost-benefit analysis to justify the amount spent; to prove that the benefits outweigh the cost
5. To formulate a basis for making rational decisions about future training plans
6. To justify the role of training for budget purposes and in cutback situations of a budget crunch.
Summarily, you are required to improve the quality of the training, concentrating on the trainers’ competency, training design, the content of the course, participant profile, expected behavioral outcomes, methods used, length of training, achievement of the training objectives through improving the training delivery as a whole. Knowing the purposes of training evaluation, you will now be acquainted with the process through evaluation.
Principles of Training Evaluation:
1. Evaluation specialists must be clear of the training program and also about the goals and purposes of evaluation.
2. Evaluation should be continuous.
3. The evaluation must be specific.
4. The evaluation must provide the means and focus for trainers to be able to appraise themselves, their practices, and their products.
5. The evaluation must be based on objective methods and standards.
6. Realistic target dates must be set for each phase of the evaluation process.
Training Evaluation – 2 Important Methods for Evaluating Training: ROI of Training and Training Hours per Employee (With Formula)
The various methods of training evaluation are observation, questionnaires, interview, cost-benefit analysis, self diaries, and self-recording of specific incidents.
Method # 1. ROI of Training:
About $5.6 billion to $16.8 billion is wasted annually on ineffective training Programmes. “American industry is spending billions and billions on training programs and not evaluate their effectiveness. You have to measure it.”- Cary Cherniss, Rutgers University (2017).
Employee training is responsible for productivity improvements, greater workforce flexibility, savings on material and capital costs, improved quality of the final product or service, and a more motivated workforce (Santana, 2019). To know these it is necessary to calculate the Rate of Return on a firm’s Investment (ROI) of training.
There is no other workplace issue on which so much money is spent with as little accountability as training.
Alternative Training Evaluation ROI Methods:
Alternatives to ROI are increasingly being used to measure the success of training programs.
Some of these methods include the following:
1. 360-Degree Appraisal Feedback Process – This performance evaluation system uses input from all employee levels (appraisers, supervisors, and peers) to assess performance (Santana, 2019).
2. Performance-Learning-Satisfaction Evaluation – This method uses both conceptual components and data processing to assess business results at an organizational level and financial results in terms of monetary ratios. Its creators claim the model has a 2:1 return on investment (Santana, 2019).
3. The Balanced Scorecard – This method tracks the key elements of a company’s strategy using both financial and operational measures. The balanced scorecard considers four primary questions- How does the company look to the shareholders? (Financial perspective); How do the customers see the company? (Customer perspective); at what must the company excel? (Internal perspective); Can the company continue to improve and create value? (Learning perspective) (Santana, 2019).
Research also highlights other methods to creatively measure training’s ROI.
Such methods include the following:
1. The Success Case Method—Demonstrates business value through credible stories that show the economic impact on the company. This method relies on short, behavior-based questionnaires that trainees complete post-training (Santana, 2019). The survey is designed to test some fundamentals about what the learning was supposed to deliver. Low success cases help the training department determine whether it is the training program or an unrelated issue that may have caused the low impact (Santana, 2019).
2. Show the Payback—Measures the payback period of the training initiative to show how long it will take to recover the investment the organization initially made (Santana, 2019). Two major drawbacks are that such calculations ignore the opportunity cost and the time value of money.
3. Evaluate the Consequences of Not Training—Demonstrates how not training the workforce will impact the organization in terms of competition, productivity, product marketing, and equal opportunity violations (Santana, 2019). The evaluator must identify a potential loss or problem and predict its financial or business impact while isolating how the lack of performance will have an impact, as well as the non-performance factors and their costs. Such costs are then compared to the benefits of the training program in terms of estimated costs (Santana, 2019). This method can be somewhat subjective and is not appropriate for all training programs.
4. 80/20 Rule—Estimates the soft benefits like customer satisfaction with 80 percent accuracy using only 20 percent of the effort (Santana, 2019). Testing and surveys can measure soft benefits like morale and increased employee satisfaction as a result of the training program.
5. Merck and Company use a unique formula, detailed below, to measure the impact of specific training programs (Santana, 2019). With this model, Merck determined that its average ROI for training programs is 84 percent and terminated 53 programs that were not producing high enough returns.
GAIN = D x SD$ x JSI x N
Where,
D = shift in performance by average individual undergoing training expressed in standard deviations from pre-training average,
SD$ = the value in dollars of one standard deviation of performance shift,
JSI = percentage of job skills affected by training.
N = number of participants who underwent training.
Method # 2. Training Hours per Employee (THPE):
This is a measure of the average number of hours spent by an organization’s employees on training activities.
The number of training hours provides a good gauge of the company’s overall training efforts. It is an indirect measure for assessing the value the organization places on its workers.
Training Evaluation – Kirkpatrick’s Model of Training Evaluation
Level 1 – Reaction:
Questionnaires, interviews, group discussion, or asking trainees to write a report can be used. Care must be taken with all of these methods. Very often participants have enjoyed a course, even if they learned very little (Santana, 2019). Factors such as the quality of the lunch provided, or the comfort of the chairs, may influence the assessment of the training given. The other participants may have spoilt a sound course, or conversely saved a poor course.
Trainees are not always in a position to know immediately whether what they have learned will be useful and it may be best to wait some considerable time before asking for an opinion (Santana, 2019). Sometimes a trainee may have felt unfairly criticized during a course, and so may ‘rubbish’ it in retaliation. The more training a person receives, the more critical he or she is likely to become. Standards and expectations rise with experience (Santana, 2019).
Using more than one technique can be helpful to gain a broader picture. Also, look out for cues such as an increase or decrease in demand for the training (where there is a choice), or if the line managers start asking for one particular trainer in preference to another (Santana, 2019).
Level 2 – Learning:
Tests, examinations, workplace-based assessments of competence, projects, or attitude questionnaires are the key techniques here (Santana, 2019). Some learning situations are easy to test for (e.g typing ability), whereas others necessarily involve a good deal of subjectivity (e.g counseling skills).
Yet other learning is so long-term in its nature that direct methods are frankly not appropriate. For example, if a newly appointed supervisor attends a course, then an end test or examination can only tell us if he or she has learned certain terms, concepts, or models. It cannot tell us if he or she will become a good supervisor by applying that learning in the work situation.
The processes used at level 2 are often termed validation (Santana, 2019).
Level 3 – Behaviour:
This level requires an assessment of improved performance on the job. This is easiest in jobs where before-and-after measures can easily be made (e.g the speed at which an insurance proposal form can be processed).
It becomes more difficult to evaluate performance in jobs that are less prescribed and where measurement is imprecise (e.g training design). There may be a time-lag between training and the appearance of indicators of performance improvement (Santana, 2019).
For instance, upon returning to work after attending a course on leadership, a manager may immediately practice what he or she has learned – but the results of this take two or three months to become apparent (Santana, 2019).
During that time other factors in the situation may have changed – there may have been some new staff recruited, or some redundancies have affected morale. If we were to instigate a long-term assessment process, we would also find it difficult to separate the influence of day-to-day experience from the influence of the formal training course, it is often impossible to isolate the precise influence of the training. Often the trainer has to resort to indirect performance assessment measures to gauge the influence of the training.
Level 4 – Results:
Because departmental and organizational results depend upon many people and it is difficult to attribute improvements to the efforts of specific individuals, evaluation at this level often has to be conducted in a more general way (Santana, 2019).
Does the overall training program result in greater efficiency, profitability, or whatever? If we were to try to look at the impact of a large training program on a part of a large organization, we can take an experimental approach (Santana, 2019).
Ideally, we take two identical units. One is given lots of training, the other is given none. Two years later, the difference in performance is apparent!
Such an approach is not one that can be easily advocated. If we believe that the training is likely to be of value, it is unfair, perhaps even unethical, to withhold it from one of the units to experiment.
However, it is sometimes possible to obtain historical information that shows a correlation between spending (or some other measure) on training and organizational performance (Santana, 2019). Perhaps two similar units within the same organization can be compared and the relationship between past training activity and other measures can be assessed (e.g. accident rate, machine downtime, customer complaints).
Training Evaluation – How to Evaluate the Effectiveness of a Training Programme?
There are several methods of evaluating the effectiveness of a training program but the following methods are important:
1. Assessment of trainers’ comments and reactions to the training program after the training is over.
2. Observation of trainees during the training program.
3. Comparing the on-the-job performance of the trainees before and after training.
4. Collection of opinions and judgments of trainers, superiors, and peers.
5. Giving oral and written tests to trainees to find out how far they have learned through the training program.
6. Cost-benefit of the training program.
7. Measurements of levels in employees’ absenteeism, turnover, productivity, wastage or scrap of materials, accidents, breakage of machinery during the pre-training period and post-training period.
8. Evaluation of trainees’ skill level before and after training.
9. Collection of opinions of the trainees’ subordinates regarding their job performance and behavior.
10. Collection of information through evaluation forms duly filled up by the trainees.
11. Knowing trainees’ expectations before training and collecting their views regarding the attainment of the expectations after training.
Management can make training program effective if the following guidelines are strictly followed:
1. Training objective to be specific to performance standards of trainees to achieve organizational objectives
2. Proper screening of personal needs in comparison to operational needs will give an added advantage to establish the actual needs of individual employees.
3. Efforts should be made to establish where the trainee has the required intelligence, maturity, and also the motivation to complete the training program. If this is not noticed among trainees the training program may be postponed or canceled till adequate improvements are noticed.
4. The trained should be encouraged to see the training by making him aware of the personal benefits he can avail through better performance.
5. Training program to be planned so that it is related to the trainees’ previous experience and background. This background may be used as a base for new advancement in a professional career.
6. Efforts to be taken to create a friendly atmosphere that is conducive to a good learning environment. Any deviation in the process of the environment to be avoided well in advance. Permission to be obtained from top-level management before commencing the training at lower levels.
7. It is to be understood that all levels of trainees do not progress at the same speed; hence, flexibility should be shown in judging the rate of progress in the training program.
8. The personal involvement of trainees, as far as possible be encouraged in the training program. He should be allowed to participate to have adequate practical knowledge in the newly needed behavioral norms.
9. As the trainee’s acquisition knowledge, skills, attributes, and utilize them while executing the job. At the same time, the trainee should be motivated for better performance and also be given necessary incentives.
10. Trainees to be given personal guidance as and when needed to help obviate learning obstacles.
Conclusion
In conclusion, if excellent training results are a top priority for your organization, then there is a need to find ways to continuously improve your training program. Using the steps in this article start by deciding on a method, then select your measurements, and choose the right training evaluation tools. Once you’ve set up the right method for your organization, it will be much easier to evaluate and improve your employee training program.
Milton Jack is a Business Consultant at Industrial Psychology Consultants (Pvt) Ltd, a business management and human resources consulting firm.
LinkedIn: linkedin.com/in/milton-jack-9798b966
Phone: +263 242 481946-48/481950
Mobile: +263 774 730 913
Email: milton@ipcconsultants.com
Main Website: www.ipcconsultants.com