There are many measures used in the evaluation of training.
The problem is many of these statements are primarily measures of Inputs into the program, not Outputs or Outcomes to be achieved, let alone, the Impact as a consequence of the training program.
In addition, training evaluation is often based on ‘feel good’ questions or statements that add no value to the understanding of what training outcomes were achieved.
Here are 10 measures that are typical and widely used. Note the ‘feel good’ statements.
- I understand the purpose of the training delivered;
- Training material presented was in accordance with the stated learning objectives;
- The depth of material presented was in accordance with my expectations;
- The structure of the training material was easy to understand;
- The knowledge training helped me understand the material;
- My horizons were enriched with the experiences shared by the trainer;
- I feel comfortable with the way the trainer presented the program;
- I was given the opportunity to practice my newly acquired skills in the classroom;
- I feel comfortable with the training facilities provided;
- I was satisfied with this training program.
Outcomes-driven training evaluation is driven by metrics that reflect the outcomes delivered against those prescribed for the course, thus, assessment of training programs within an outcomes framework will follow a Program Logic Model. A logic model identifies the process and outcome portions of an activity and shows the relationship between inputs and expected outcomes and provides a graphic summary of how job components relate to the whole. The Program Logic Model below is used as a tool to show the relationships between the activities a job and the Outcomes to be delivered.
Inputs > Outputs (Activities & Participation) > Outcomes (Short, Medium & Long-term) > Impact
Outcomes Based Evaluation
Outcome based evaluation (OBE) is a systematic way to determine if a program or project has achieved its goals. The organized process of developing a project using OBE helps to establish clear benefits (outcomes), to measure those benefits (indicators), clarify the individuals or groups for which the project’s benefits are intended (target audience). OBE will answer:
- Who is your target audience?
- What kinds of resources are needed?
- When are the resources needed?
- Where do I get the resources?
- What actions do I need to take?
- How did my patrons benefit?
The Difference between Outputs and Outcomes
- Outputs are measures of the volume of a project’s activity: products created or delivered, people served, activities and services carried out; the “things” of a project. They are almost always expressed in numerical format.
- Outcomes are the “people” part of the project, that is, what was the change or benefit to people because of the outputs. They are almost always expressed as a statement of change or benefit to your target audience.
Inputs are resources a program uses in delivering the Outputs/Outcomes – examples include:
- Staff and staff time, money spent for training, training specialists, guest knowledge leaders’ time, training facilities, including equipment and supplies.
Activities are what the job holders do to fulfill the requirements of the job in achieving the organization’s mission, thus job activities result in outputs; for example:
- Provide specific job training, management leadership training, educate the public about the need for tax compliance, counsel bad tax payers, create mentoring relationships for colleagues/tax agents.
Outputs are direct products of an employee’s activities. They are quantitative and are typically measured by how many?; how often?; over what duration?
Outputs Indicators include:
- Number of reports produced, the quality of the reports, number of tax payer/tax agent counselling-mentoring sessions conducted, number of tax educational materials distributed and the hours of service delivered.
Outcomes are the short-, intermediate- or long-term deliverables an employee produces and include:
- New marketing knowledge resulting and improved customer experience or making more profitable sales, changed practices in the sales/marketing process or modified work behavior of subordinates and altered workflow practices.
Outcome Indicators are the specific characteristics or behaviors measured to track an employees’ success in achieving the outcomes delivered.
- Outcomes are observable: “can you see it, hear it or read it?”
For an outcome that states “improvement in leadership skills”, some indicators might be:
- Improved employee/subordinate interaction skills, reduction in workplace conflict, enhanced engagement with 360 degree working relationships.
For an outcome identified as “leadership style”, some indicators might be:
- Engaged listening (not cutting in on a colleague’s/subordinate’s explanation);
- Maintaining a recommended level of blood pressure in stressful situations; and
- Empowering subordinates to achieve their assigned tasks.
Outcome Targets are numerical objectives for a job holder’s level of Outcomes achievement.
After a performance appraisal has collected at least one round of outcome data, it can use that data to set targets for the number and per cent of deliverables to be achieved in the next reporting period.
Outcome measurement is the regular, systematic tracking of the extent to which a job holders delivers the outcomes intended.
Impact is the ultimate impact(s) expected to result from the job-holders activities.
They are long-term changes which the corporation expects the job holder to make in delivering against the corporate objectives in their area of responsibility.
- They are broad statements of intended achievements.
Examples include: enhanced sales and client engagement.
The Logic Model is a step-by-step approach for defining and measuring outcomes. It is an evaluation framework for your training evaluation project.
How does this Logic Model fit with your current training evaluation process?